{"id":13487852,"url":"https://github.com/inbarhub/DDPM_inversion","last_synced_at":"2025-03-27T23:31:51.207Z","repository":{"id":175929134,"uuid":"629968895","full_name":"inbarhub/DDPM_inversion","owner":"inbarhub","description":"Official pytorch implementation of the paper: \"An Edit Friendly DDPM Noise Space: Inversion and Manipulations\". CVPR 2024. ","archived":false,"fork":false,"pushed_at":"2024-07-11T21:46:22.000Z","size":34855,"stargazers_count":271,"open_issues_count":5,"forks_count":12,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-30T23:35:54.888Z","etag":null,"topics":["ddpm","editing","generative-model","inversion","latent-space","text-guided-image-editing","text-guided-image-manipulation"],"latest_commit_sha":null,"homepage":"https://inbarhub.github.io/DDPM_inversion/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/inbarhub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-19T11:53:30.000Z","updated_at":"2024-10-30T19:15:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"deafbe2d-4364-478a-be12-4f6ee131a104","html_url":"https://github.com/inbarhub/DDPM_inversion","commit_stats":null,"previous_names":["inbarhub/ddpm_inversion"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inbarhub%2FDDPM_inversion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inbarhub%2FDDPM_inversion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inbarhub%2FDDPM_inversion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inbarhub%2FDDPM_inversion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/inbarhub","download_url":"https://codeload.github.com/inbarhub/DDPM_inversion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245944020,"owners_count":20697945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ddpm","editing","generative-model","inversion","latent-space","text-guided-image-editing","text-guided-image-manipulation"],"created_at":"2024-07-31T18:01:05.139Z","updated_at":"2025-03-27T23:31:47.445Z","avatar_url":"https://github.com/inbarhub.png","language":"Python","funding_links":[],"categories":["Diffusion Models Inversion","Training and Finetuning Free"],"sub_categories":["Train-Free","Training and Finetuning Free: Inversion/Sampling Modification"],"readme":"\u003c!-- [![DDPM inversion](https://img.shields.io/badge/single%20image-generative%20model-yellow)](https://github.com/topics/single-image-generation) --\u003e\n[![Python 3.8](https://img.shields.io/badge/python-3.812+-blue)](https://www.python.org/downloads/release/python-38/)\n[![torch](https://img.shields.io/badge/torch-2.0.0+-green)](https://pytorch.org/)\n\n\n# DDPM inversion, CVPR 2024\n\n[Project page](https://inbarhub.github.io/DDPM_inversion/) | [Arxiv](https://arxiv.org/abs/2304.06140) | [Supplementary materials](https://inbarhub.github.io/DDPM_inversion/resources/inversion_supp.pdf) | [Hugging Face Demo](https://huggingface.co/spaces/LinoyTsaban/edit_friendly_ddpm_inversion)\n### Official pytorch implementation of the paper: \u003cbr\u003e\"An Edit Friendly DDPM Noise Space: Inversion and Manipulations\"\n#### Inbar Huberman-Spiegelglas, Vladimir Kulikov and Tomer Michaeli \n\u003cbr\u003e\n\n![](imgs/teaser.jpg)\nOur inversion can be used for text-based **editing of real images**, either by itself or in combination with other editing methods.\nDue to the stochastic nature of our method, we can generate **diverse outputs**, a feature that is not naturally available with methods relying on the DDIM inversion.\n\nIn this repository we support editing using our inversion, prompt-to-prompt (p2p)+our inversion, ddim or [p2p](https://github.com/google/prompt-to-prompt) (with ddim inversion).\u003cbr\u003e\n**our inversion**: our ddpm inversion followed by generating an image conditioned on the target prompt. \n\n**prompt-to-prompt (p2p) + our inversion**: p2p method using our ddpm inversion. \n\n**ddim**: ddim inversion followed by generating an image conditioned on the target prompt.\n\n**p2p**: p2p method using ddim inversion (original paper).\n\n## Table of Contents\n* [Requirements](#Requirements)\n* [Repository Structure](#Repository-Structure)\n* [Algorithm Inputs and Parameters](#Algorithm-Inputs-and-Parameters)\n* [Usage Example](#Usage-Example)\n\n* [Citation](#Citation)\n\n## Requirements \n\n```\npython -m pip install -r requirements.txt\n```\nThis code was tested with python 3.8 and torch 2.0.0. \n\n## Repository Structure \n```\n├── ddm_inversion - folder contains inversions in order to work on real images: ddim inversion as well as ddpm inversion (our method).\n├── example_images - folder of input images to be edited\n├── imgs - images used in this repository readme.md file\n├── prompt_to_prompt - p2p code\n├── main_run.py - main python file for real image editing\n└── test.yaml - yaml file contains images and prompts to test on\n```\n\nA folder named 'results' will be automatically created and all the results will be saved to this folder. We also add a timestamp to the saved images in this folder.\n\n## Algorithm Inputs and Parameters\nMethod's inputs: \n```\ninit_img - the path to the input images\nsource_prompt - a prompt describing the input image\ntarget_prompts - the edit prompt (creates several images if multiple prompts are given)\n```\nThese three inputs are supplied through a YAML file (please use the provided 'test.yaml' file as a reference).\n\n\u003cbr\u003e\nMethod's parameters are:\n\n```\nskip - controlling the adherence to the input image\ncfg_tar - classifier free guidance strengths\n```\nThese two parameters have default values, as descibed in the paper.\n\n## Usage Example \n```\npython3 main_run.py --mode=\"our_inv\" --dataset_yaml=\"test.yaml\" --skip=36 --cfg_tar=15 \npython3 main_run.py --mode=\"p2pinv\" --dataset_yaml=\"test.yaml\" --skip=12 --cfg_tar=9 \n\n```\nThe ```mode``` argument can also be: ```ddim``` or ```p2p```.\n\nIn ```our_inv``` and ```p2pinv``` modes we suggest to play around with ```skip``` in the range [0,40] and ```cfg_tar``` in the range [7,18].\n\n**p2pinv and p2p**:\nNote that you can play with the cross-and self-attention via ```--xa``` and ```--sa``` arguments. We suggest to set them to (0.6,0.2) and (0.8,0.4) for p2pinv and p2p respectively.\n\n**ddim and p2p**:\n```skip``` is overwritten to be 0.\n\n\u003c!-- ## Create Your Own Editing with Our Method\n(1) Add your image to /example_images. \u003cbr\u003e\n(2) Run ``main_run.py --mode=\"our_inv\"``, choose ``skip`` and ``cfg_tar``. \u003cbr\u003e\n\nExample:\n```\npython3 main_run.py --skip=20 --cfg_tar=10 --img_name=gnochi_mirror --cfg_src='a cat is sitting next to a mirror' --cfg_tar='a drawing of a cat sitting next to a mirror'\n```  --\u003e\n\nYou can edit the test.yaml file to load your image and choose the desired prompts.\n\n\u003c!-- ## Sources \n\nThe DDPM code was adapted from the following [pytorch implementation of DDPM](https://github.com/lucidrains/denoising-diffusion-pytorch). \n\nThe modified CLIP model as well as most of the code in `./text2live_util/` directory was taken from the [official Text2live repository](https://github.com/omerbt/Text2LIVE).  --\u003e\n \n## Citation\nIf you use this code for your research, please cite our paper:\n```\n@inproceedings{huberman2024edit,\n  title={An edit friendly {DDPM} noise space: Inversion and manipulations},\n  author={Huberman-Spiegelglas, Inbar and Kulikov, Vladimir and Michaeli, Tomer},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={12469--12478},\n  year={2024}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finbarhub%2FDDPM_inversion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finbarhub%2FDDPM_inversion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finbarhub%2FDDPM_inversion/lists"}