{"id":30351606,"url":"https://github.com/samir-atra/controlnet_implementation","last_synced_at":"2026-05-03T12:33:32.197Z","repository":{"id":308350896,"uuid":"985433603","full_name":"Samir-atra/ControlNet_Implementation","owner":"Samir-atra","description":"AN implementation of the Adding Conditional Control to Text-to-Image Diffusion Models research paper using Keras and TensorFlow","archived":false,"fork":false,"pushed_at":"2025-09-06T02:29:46.000Z","size":15745,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-06T04:19:57.289Z","etag":null,"topics":["keras","python","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Samir-atra.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-05-17T18:50:13.000Z","updated_at":"2025-09-06T02:29:49.000Z","dependencies_parsed_at":"2025-09-06T16:17:01.317Z","dependency_job_id":null,"html_url":"https://github.com/Samir-atra/ControlNet_Implementation","commit_stats":null,"previous_names":["samir-atra/controlnet_implementation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Samir-atra/ControlNet_Implementation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samir-atra%2FControlNet_Implementation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samir-atra%2FControlNet_Implementation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samir-atra%2FControlNet_Implementation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samir-atra%2FControlNet_Implementation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Samir-atra","download_url":"https://codeload.github.com/Samir-atra/ControlNet_Implementation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Samir-atra%2FControlNet_Implementation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32569712,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-03T06:36:36.687Z","status":"ssl_error","status_checked_at":"2026-05-03T06:36:09.306Z","response_time":103,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keras","python","tensorflow"],"created_at":"2025-08-18T23:09:59.736Z","updated_at":"2026-05-03T12:33:32.189Z","avatar_url":"https://github.com/Samir-atra.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ControlNet Implementation\n\nThis repository contains a Keras and TensorFlow implementation of the research paper: \"Adding Conditional Control to Text-to-Image Diffusion Models\". This project provides the building blocks for a ControlNet model, which allows for conditioning a text-to-image diffusion model on an additional input image.\n\nThis implementation is a work in progress and is based on the work from [keras-team/keras-hub/pull/2209](https://github.com/keras-team/keras-hub/pull/2209).\n\n## File Descriptions\n\n*   `LICENSE`: The license for this project.\n*   `README.md`: This file.\n*   `docs/(ControlNet) Planning-oriented Autonomous Driving.pdf`: A research paper on a related topic. The ControlNet implementation in this repository is based on the paper \"Adding Conditional Control to Text-to-Image Diffusion Models\".\n*   `script.sh`: An empty bash script.\n*   `src/`: This directory contains the source code for the ControlNet implementation.\n    *   `__init__.py`: An empty file that makes the `src` directory a Python package.\n    *   `clip_encoder.py`: Contains the `CLIPTextEncoder` class, which is used to encode text prompts into embeddings.\n    *   `controlnet.py`: Contains the `get_controlnet_model` function, which creates the ControlNet model.\n    - `sd_encoder_block.py`: Implements a U-Net-like architecture, which is the main model that ControlNet is designed to control.\n\n## Dependencies\n\nThis project requires the following Python libraries:\n\n*   `tensorflow`\n*   `keras`\n*   `keras_cv`\n*   `keras_hub`\n\nYou can install these dependencies using pip:\n\n```bash\npip install tensorflow keras keras_cv keras_hub\n```\n\n## Usage\n\nThe components in this repository can be used to build a text-to-image model that is conditioned on an additional input image. Here is an example of how you might use the `ControlNet` and `CLIPTextEncoder` models:\n\n```python\nimport tensorflow as tf\nfrom src.controlnet import get_controlnet_model\nfrom src.clip_encoder import CLIPTextEncoder\n\n# --- Parameters ---\nIMG_SIZE = (256, 256)\nPROMPT = \"a photograph of an astronaut riding a horse\"\n\n# --- Models ---\n\n# ControlNet model\ncontrolnet_model = get_controlnet_model(IMG_SIZE)\ncontrolnet_model.summary()\n\n# CLIP Text Encoder\ntext_encoder = CLIPTextEncoder()\ntext_embeddings = text_encoder([PROMPT])\n\n# --- Example Usage ---\n\n# A (dummy) conditioning image\nconditioning_image = tf.zeros((1, *IMG_SIZE, 3))\n\n# The ControlNet model takes a conditioning image and outputs a list of feature maps\ncontrol_outputs = controlnet_model(conditioning_image)\n\nprint(\"Text embeddings shape:\", text_embeddings.shape)\nprint(\"Number of control outputs:\", len(control_outputs))\nfor i, output in enumerate(control_outputs):\n    print(f\"Control output {i+1} shape:\", output.shape)\n\n```\n\nThis example demonstrates how to create the ControlNet model and the CLIP text encoder, and how to get the outputs from each. These outputs would then be injected into a larger diffusion model (like the one in `sd_encoder_block.py`) to guide the image generation process.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamir-atra%2Fcontrolnet_implementation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamir-atra%2Fcontrolnet_implementation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamir-atra%2Fcontrolnet_implementation/lists"}