{"id":13487928,"url":"https://github.com/ali-vilab/MimicBrush","last_synced_at":"2025-03-27T23:32:11.014Z","repository":{"id":243966950,"uuid":"811656881","full_name":"ali-vilab/MimicBrush","owner":"ali-vilab","description":"Official implementations for paper: Zero-shot Image Editing with Reference Imitation","archived":false,"fork":false,"pushed_at":"2024-06-15T16:17:10.000Z","size":16209,"stargazers_count":1115,"open_issues_count":10,"forks_count":78,"subscribers_count":14,"default_branch":"main","last_synced_at":"2024-10-30T23:36:09.443Z","etag":null,"topics":["aigc","customization","image-composition","image-editing","texture-transfer"],"latest_commit_sha":null,"homepage":"https://xavierchen34.github.io/MimicBrush-Page/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ali-vilab.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-07T03:24:20.000Z","updated_at":"2024-10-30T08:53:45.000Z","dependencies_parsed_at":"2024-06-15T03:40:17.063Z","dependency_job_id":"bc53ed36-f549-40dc-822e-f1c128e8ae46","html_url":"https://github.com/ali-vilab/MimicBrush","commit_stats":null,"previous_names":["ali-vilab/mimicbrush"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMimicBrush","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMimicBrush/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMimicBrush/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMimicBrush/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ali-vilab","download_url":"https://codeload.github.com/ali-vilab/MimicBrush/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245944020,"owners_count":20697945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aigc","customization","image-composition","image-editing","texture-transfer"],"created_at":"2024-07-31T18:01:06.617Z","updated_at":"2025-03-27T23:32:05.981Z","avatar_url":"https://github.com/ali-vilab.png","language":"Python","funding_links":[],"categories":["Text Guided Image Editing","Python","\u003cspan id=\"image\"\u003eImage\u003c/span\u003e","Projekte"],"sub_categories":["\u003cspan id=\"tool\"\u003eLLM (LLM \u0026 Tool)\u003c/span\u003e","🌄 Image"],"readme":"\u003cp align=\"center\"\u003e\n\n  \u003ch2 align=\"center\"\u003eZero-shot Image Editing with Reference Imitation\u003c/h2\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://xavierchen34.github.io/\"\u003e\u003cstrong\u003eXi Chen\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://scholar.google.com.hk/citations?user=mZwJLeUAAAAJ\u0026hl=zh-CN\"\u003e\u003cstrong\u003eYutong Feng\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://mengtingchen.github.io/\"\u003e\u003cstrong\u003eMengting Chen\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://openreview.net/profile?id=~Yiyang_Wang2\"\u003e\u003cstrong\u003eYiyang Wang\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://jshilong.github.io/\"\u003e\u003cstrong\u003eShilong Zhang\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://scholar.google.com/citations?user=8zksQb4AAAAJ\u0026hl=zh-CN\"\u003e\u003cstrong\u003eYu Liu\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://shenyujun.github.io/\"\u003e\u003cstrong\u003eYujun Shen\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://hszhao.github.io/\"\u003e\u003cstrong\u003eHengshuang Zhao\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr\u003e\n    \u003cbr\u003e\n        \u003ca href=\"https://arxiv.org/abs/2406.07547\"\u003e\u003cimg src='https://img.shields.io/badge/arXiv-MimicBrush-red' alt='Paper PDF'\u003e\u003c/a\u003e\n        \u003ca href='https://xavierchen34.github.io/MimicBrush-Page/'\u003e\u003cimg src='https://img.shields.io/badge/Project_Page-MimicBrush-green' alt='Project Page'\u003e\u003c/a\u003e\n        \u003ca href='https://modelscope.cn/studios/iic/mimicbrush-demo/summary'\u003e\u003cimg src='https://img.shields.io/badge/ModelScope-MimicBrush-yellow'\u003e\u003c/a\u003e\n        \u003ca href='https://huggingface.co/spaces/xichenhku/MimicBrush'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue'\u003e\u003c/a\u003e\n    \u003cbr\u003e\n    \u003cb\u003eThe University of Hong Kong \u0026nbsp; | \u0026nbsp;  Alibaba Group  | \u0026nbsp;  Ant Group \u003c/b\u003e\n  \u003c/p\u003e\n  \n  \u003ctable align=\"center\"\u003e\n    \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"assets/teaser.png\"\u003e\n    \u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/table\u003e\n\n## News\n* **[2024.06.12]** Release inference code, local gradio demo, online demo.\n* **[Todo]** Release our benchmark.\n\n## Community Contributions\n[ComfyUI version](https://github.com/AIFSH/ComfyUI-MimicBrush) by [@AIFSH](https://github.com/AIFSH)  \n\n## Installation\nInstall with `conda`: \n```bash\nconda env create -f environment.yaml\nconda activate mimicbrush\n```\nor `pip`:\n```bash\n#Python==3.8.5\npip install -r requirements.txt\n```\n\n\n## Download Checkpoints\nDownload SD-1.5 and SD-1.5-inpainting checkpoint: \n* You could download them from HuggingFace [stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5) and [stable-diffusion-inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting/)\n* However, the repo above contains many models that would not be used, we provide a clean version at [cleansd](https://modelscope.cn/models/xichen/cleansd/)\n\nDownload MimicBrush checkpoint, along with a VAE, a CLIP encoder, and a depth model\n* Download the weights on ModelScope [xichen/MimicBrush](https://www.modelscope.cn/models/xichen/MimicBrush)\n* The model is big because it contains two U-Nets.\n\nYou could use the following code to download them from modelscope\n\n```python\nfrom modelscope.hub.snapshot_download import snapshot_download as ms_snapshot_download\n\nsd_dir = ms_snapshot_download('xichen/cleansd', cache_dir='./modelscope')\nprint('=== Pretrained SD weights downloaded ===')\nmodel_dir = ms_snapshot_download('xichen/MimicBrush', cache_dir='./modelscope')\nprint('=== MimicBrush weights downloaded ===')\n```\nor from Huggingface\n\n```python\nfrom huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"xichenhku/cleansd\", local_dir=\"./cleansd\")\nprint('=== Pretrained SD weights downloaded ===')\nsnapshot_download(repo_id=\"xichenhku/MimicBrush\", local_dir=\"./MimicBrush\")\nprint('=== MimicBrush weights downloaded ===')\n```\n\n\n## Gradio Demo \nFirst, modify `./configs/inference.yaml` to set the path of model weight. Afterwards, run the script:\n```bash\npython run_gradio3_demo.py\n```\n\nThe gradio demo would look like the UI shown below. \n\n\u003cfont color=\"#dd0000\"\u003e*Please do not forget to click ''keep the original shape'' if you want condut texture transfer like the third case.\u003c/font\u003e\u003cbr /\u003e \n\nA biref tutorial:\n\n* Upload/select a source image to edit. \n* Draw the to-edit regionon the source image.\n* Upload/select a reference image. \n* Run. \n\n\u003ctable align=\"center\"\u003e\n  \u003ctr\u003e\n  \u003ctd\u003e\n    \u003cimg src=\"assets/demo.png\"\u003e\n  \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Inference\n\n1. Dowload our evaluation benchmark at Google Drive:\n    * URL: [to be released]\n\n2. Set the path to each dataset and checkpoints in `./config/inference.yaml` :\n\n3. Run inference with \n    ```bash\n    python run_inference_benchmark.py\n    ```\n\n\n## Acknowledgements\nThis project is developped on the codebase of [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter) and [MagicAnimate](https://github.com/magic-research/magic-animate)  . We  appreciate this great work! \n\n\n## Citation\nIf you find this codebase useful for your research, please use the following entry.\n```BibTeX\n@article{chen2024mimicbrush,\n  title={Zero-shot Image Editing with Reference Imitation},\n  author={Chen, Xi and Feng, Yutong and Chen, Mengting and Wang, Yiyang, and Zhang, Shilong and Yu, Liu and Shen, Yujun and Zhao, Hengshuang},\n  journal={arXiv preprint arXiv:2406.07547},\n  year={2024}\n}\n```\n## Star History\n[![Star History Chart](https://api.star-history.com/svg?repos=ali-vilab/MimicBrush\u0026type=Date)](https://star-history.com/#ali-vilab/MimicBrush\u0026Date)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fali-vilab%2FMimicBrush","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fali-vilab%2FMimicBrush","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fali-vilab%2FMimicBrush/lists"}