{"id":22805056,"url":"https://github.com/datacte/prompt_injection","last_synced_at":"2025-08-20T12:30:39.132Z","repository":{"id":242247506,"uuid":"809071204","full_name":"DataCTE/prompt_injection","owner":"DataCTE","description":null,"archived":false,"fork":false,"pushed_at":"2024-06-21T12:56:43.000Z","size":24,"stargazers_count":82,"open_issues_count":1,"forks_count":19,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-12-12T10:11:57.571Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataCTE.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-01T15:51:21.000Z","updated_at":"2024-12-12T07:49:01.000Z","dependencies_parsed_at":"2024-06-15T17:25:44.976Z","dependency_job_id":"026f5c19-4b71-4454-b313-19ca89cbb064","html_url":"https://github.com/DataCTE/prompt_injection","commit_stats":null,"previous_names":["datacte/prompt_injection"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2Fprompt_injection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2Fprompt_injection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2Fprompt_injection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataCTE%2Fprompt_injection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataCTE","download_url":"https://codeload.github.com/DataCTE/prompt_injection/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230423559,"owners_count":18223435,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-12T10:12:28.630Z","updated_at":"2024-12-19T11:10:36.252Z","avatar_url":"https://github.com/DataCTE.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Prompt Injection Node for ComfyUI\n\nThis custom node for ComfyUI allows you to inject specific prompts at specific blocks of the Stable Diffusion UNet, providing fine-grained control over the generated image. It is based on the concept that the content/subject understanding of the model is primarily contained within the MID0 and MID1 blocks, as demonstrated in the B-Lora (Content Style implicit separation) paper.\nFeatures\n\nInject different prompts into specific UNet blocks\nThree different node variations for flexible workflow integration\nCustomize the learning rate of specific blocks to focus on content, lighting, style, or other aspects\nPotential for developing a \"Mix of Experts\" approach by swapping blocks on-the-fly based on prompt content\n\n# Usage\n\nAdd the prompt_injection.py node to your ComfyUI custom nodes directory\nIn your ComfyUI workflow, connect the desired node variation based on your input preferences\nSpecify the prompts for each UNet block you want to customize\nConnect the output to the rest of your workflow and generate the image\n\n# Node Variations\n\nPrompt Injection (Single Prompt): Injects a single prompt into the specified UNet blocks\nPrompt Injection (Multiple Prompts): Allows injecting different prompts into each specified UNet block\nPrompt Injection (Prompt Dictionary): Accepts a dictionary of block names and their corresponding prompts\n\n# Example\nInjecting the prompt \"white cat\" into the OUTPUT0 and OUTPUT1 blocks, while using the prompt \"blue dog\" for all other blocks, results in an image with the composition of the \"blue dog\" prompt but with a cat as the subject/content.\nAcknowledgements\n\nModified and simplified version of the node from: https://github.com/pamparamm/sd-perturbed-attention\nInspired by discussions and findings shared by @Mobioboros and @DataVoid\n\n# Future Work\n\nInvestigate the location of different concepts (e.g., lighting) within the UNet blocks\nDevelop a \"guts diagram\" of the SDXL UNet to understand where each aspect is stored\nExplore the use of different learning rates for specific blocks during fine-tuning or LoRA training\nImplement a \"Mix of Experts\" approach by swapping blocks on-the-fly based on prompt content\n\nFeel free to contribute, provide feedback, and share your findings!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacte%2Fprompt_injection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatacte%2Fprompt_injection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatacte%2Fprompt_injection/lists"}