{"id":19307021,"url":"https://github.com/picsart-ai-research/zero-painter","last_synced_at":"2025-04-12T09:42:55.068Z","repository":{"id":243301895,"uuid":"779279870","full_name":"Picsart-AI-Research/Zero-Painter","owner":"Picsart-AI-Research","description":"🔥 [CVPR 2024] The official repo for Zero-Painter!","archived":false,"fork":false,"pushed_at":"2024-06-08T10:16:45.000Z","size":3599,"stargazers_count":66,"open_issues_count":3,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-26T04:41:25.934Z","etag":null,"topics":["computer-vision","cvpr2024","generative-ai","zero-painter"],"latest_commit_sha":null,"homepage":"http://arxiv.org/abs/2406.04032","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Picsart-AI-Research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-29T13:10:39.000Z","updated_at":"2025-03-20T02:39:21.000Z","dependencies_parsed_at":"2024-06-07T22:30:37.361Z","dependency_job_id":"c8d0cdcd-8c6b-4f57-8a2f-2f7460883529","html_url":"https://github.com/Picsart-AI-Research/Zero-Painter","commit_stats":null,"previous_names":["picsart-ai-research/zero-painter"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Picsart-AI-Research%2FZero-Painter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Picsart-AI-Research%2FZero-Painter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Picsart-AI-Research%2FZero-Painter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Picsart-AI-Research%2FZero-Painter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Picsart-AI-Research","download_url":"https://codeload.github.com/Picsart-AI-Research/Zero-Painter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248549760,"owners_count":21122925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","cvpr2024","generative-ai","zero-painter"],"created_at":"2024-11-10T00:09:17.871Z","updated_at":"2025-04-12T09:42:55.048Z","avatar_url":"https://github.com/Picsart-AI-Research.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis [CVPR 2024]\n\nThis repository is the official implementation of [Zero-Painter](https://arxiv.org/abs/2406.04032).\n\n\n**[Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis](https://arxiv.org/abs/2406.04032)**\n\u003c/br\u003e\nMarianna Ohanyan*,\nHayk Manukyan*,\nZhangyang Wang,\nShant Navasardyan,\n[Humphrey Shi](https://www.humphreyshi.com)\n\u003c/br\u003e\n\n[Arxiv](https://arxiv.org/abs/2406.04032) \n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"__assets__/github/teaser.png\" width=\"800px\"/\u003e  \n\u003cbr\u003e\n\u003cem\u003e\nWe present \u003cstrong\u003eZero-Painter\u003c/strong\u003e , a novel training-free framework for layout-conditional text-to-image synthesis that facilitates the creation of detailed and controlled imagery from textual prompts. Our method utilizes object masks and individual descriptions, coupled with a global text prompt, to generate images with high fidelity. Zero-Painter employs a two-stage process involving our novel \u003cstrong\u003ePrompt-Adjusted Cross-Attention (PACA) and Region-Grouped Cross-Attention (ReGCA)\u003c/strong\u003e blocks, ensuring precise alignment of generated objects with textual prompts and mask shapes. Our extensive experiments demonstrate that Zero-Painter surpasses current state-of-the-art methods in preserving textual details and adhering to mask shapes.\n\n\u003c/em\u003e\n\u003c/p\u003e\n\n## 🔥 News\n- [2024.06.6] ZeroPainter paper and code is released.\n- [2024.02.27] Paper is accepted to CVPR 2024.\n\n\n## ⚒️ Installation\n\n\u003c!-- Install with `conda`: \n```bash\nconda env create -f environment.yaml\nconda activate zero-painter\n``` --\u003e\nInstall with `pip`:\n```bash\npip3 install -r requirements.txt\n```\n\n## 💃 Inference: Generate images with Zero-Painter\n\n1. Download [models](https://huggingface.co/PAIR/Zero-Painter) and put them in the `models` folder.\n2. You can use the following script to perform inference on the given mask and prompts pair:\n```\npython zero_painter.py \\\n  --mask-path data/masks/1_rgb.png \\\n  --metadata data/metadata/1.json \\\n  --output-dir data/outputs/\n```\n\n`meatadata` sould be in the following format\n```\n[{\n    \"prompt\": \"Brown gift box beside red candle.\",\n    \"color_context_dict\": {\n        \"(244, 54, 32)\": \"Brown gift box\",\n        \"(54, 245, 32)\": \"red candle\"\n    }\n}]\n```\n\u003c!-- ```\npython hd_inpaint.py \\\n  --model-id ONE_OF[ds8_inp, sd2_inp, sd15_inp] \\\n  --method ONE_OF[baseline, painta, rasg, painta+rasg] \\\n  --image-path HR_IMAGE_PATH \\\n  --mask-path HR_IMAGE_MASK \\\n  --prompt PROMPT_TXT \\\n  --output-dir OUTPUT_DIRECTORY\n``` --\u003e\n\n## Method\n\n\u003cimg src=\"__assets__/github/method_arch.png\" raw=true\u003e\n\n---  \n\n## 🎓 Citation\nIf you use our work in your research, please cite our publication:\n```\n@article{Zeropainter,\ntitle={Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis},\nurl={http://arxiv.org/abs/2406.04032},\npublisher={arXiv},\nauthor={Ohanyan, Marianna and Manukyan, Hayk and Wang, Zhangyang and Navasardyan, Shant and Shi, Humphrey},\nyear={2024}}\n\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpicsart-ai-research%2Fzero-painter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpicsart-ai-research%2Fzero-painter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpicsart-ai-research%2Fzero-painter/lists"}