{"id":13487931,"url":"https://github.com/yandex-research/invertible-cd","last_synced_at":"2026-04-11T03:57:29.756Z","repository":{"id":244655073,"uuid":"815860615","full_name":"yandex-research/invertible-cd","owner":"yandex-research","description":"[NeurIPS'2024] Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps","archived":false,"fork":false,"pushed_at":"2024-07-04T06:28:02.000Z","size":25101,"stargazers_count":91,"open_issues_count":2,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-13T02:37:22.177Z","etag":null,"topics":["consistency-distillation-stable-diffusion","image-editing","image-generation"],"latest_commit_sha":null,"homepage":"https://yandex-research.github.io/invertible-cd/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yandex-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-16T11:10:23.000Z","updated_at":"2025-02-25T12:18:26.000Z","dependencies_parsed_at":"2024-06-21T09:02:55.923Z","dependency_job_id":"1fa6c7b9-8f6a-41f0-94c9-b91e8a7eba29","html_url":"https://github.com/yandex-research/invertible-cd","commit_stats":null,"previous_names":["yandex-research/invertible-cd"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-research%2Finvertible-cd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-research%2Finvertible-cd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-research%2Finvertible-cd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-research%2Finvertible-cd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yandex-research","download_url":"https://codeload.github.com/yandex-research/invertible-cd/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245944020,"owners_count":20697945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consistency-distillation-stable-diffusion","image-editing","image-generation"],"created_at":"2024-07-31T18:01:06.678Z","updated_at":"2026-04-11T03:57:24.725Z","avatar_url":"https://github.com/yandex-research.png","language":"Python","funding_links":[],"categories":["Text Guided Image Editing"],"sub_categories":[],"readme":"# Invertible Consistency Distillation for \u003cbr\u003e Text-Guided Image Editing in Around 7 Steps\n\n\u003ca href='https://arxiv.org/abs/2406.14539'\u003e\u003cimg src='https://img.shields.io/badge/ArXiv-PDF-red'\u003e\u003c/a\u003e \u0026nbsp; \n\u003ca href='https://yandex-research.github.io/invertible-cd/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e \u0026nbsp; \n\u003ca href=\"https://huggingface.co/spaces/dbaranchuk/iCD-image-generation\"\u003e\n\t    \u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Demo-Generation-orange' /\u003e\n\u003c/a\u003e\u0026nbsp;\n\u003ca href=\"https://huggingface.co/spaces/dbaranchuk/iCD-image-editing\"\u003e\n\t    \u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Demo-Editing-orange' /\u003e\n\u003c/a\u003e\u0026nbsp;\n\nThis paper proposes **invertible Consistency Distillation**, enabling\n\n1. highly efficient and accurate **text-guided image editing**\n2. diverse and high-quality **image generation**\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/main.jpg\" width=\"1080px\"/\u003e\n\u003c/p\u003e\n\n## Table of contents\n\n* [Installation](#installation)\n* [Easy-to-run examples](#easy-to-run-examples) (iCD-SD1.5)\n    * [Generation](#generation-with-sd15)\n    * [Editing](#editing-with-sd15)\n* [Easy-to-run examples](running/sdxl) (iCD-SDXL)\n* [In-depth generation and editing](running) (iCD-SDXL and iCD-SD1.5)\n* [iCD training example](training) (iCD-SDXL and iCD-SD1.5)\n* [Citation](#citation)\n\n## Installation\n\n```shell\n# Clone a repo\ngit clone https://github.com/yandex-research/invertible-cd\n\n# Create an environment and install packages\nconda create -n icd python=3.10 -y \nconda activate icd\n\npip3 install -r requirements/req.txt\n```\nWe provide the following checkpoints:\n\n1. Guidance distilled diffusion models\n    - [Stable Diffusion 1.5, 3GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/sd15_cfg_distill.pt.tar.gz\n      )\n    - [SDXL, 8.9GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/sdxl_cfg_distill.pt.tar.gz)\n\nThese models saved as `.pt` files.\n\n2. Invertible Consistency Distillation (_forward_ and _reverse_ CD) on top of the guidance distilled models\n\n| Model                                                                                                     | Steps | Time steps                                                       |\n|-----------------------------------------------------------------------------------------------------------|-------|------------------------------------------------------------------|\n| [iCD-SD1.5, 0.5GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/iCD-SD15_4steps_1.tar.gz)        | 4     | Reverse: [259, 519, 779, 999]; \u003cbr\u003e Forward: [19, 259, 519, 779] |\n| [iCD-SD1.5, 0.5GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/iCD-SD15_4steps_2.tar.gz)        | 4     | Reverse: [249, 499, 699, 999]; \u003cbr\u003e Forward: [19, 249, 499, 699] |\n| [iCD-SD1.5, 0.5GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/iCD-SD15_3steps.tar.gz)         | 3     | Reverse: [339, 699, 999]; \u003cbr\u003e Forward: [19, 339, 699]           |\n| [iCD-SDXL, 1.4GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/iCD-SDXL_4steps_1.tar.gz) | 4     | Reverse: [259, 519, 779, 999]; \u003cbr\u003e Forward: [19, 259, 519, 779] |\n| [iCD-SDXL, 1.4GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/iCD-SDXL_4steps_2.tar.gz) | 4     | Reverse: [249, 499, 699, 999]; \u003cbr\u003e Forward: [19, 249, 499, 699] |\n| [iCD-SDXL, 1.4GB](https://storage.yandexcloud.net/yandex-research/invertible-cd/iCD-SDXL_3steps.tar.gz)   | 3     | Reverse: [339, 699, 999]; \u003cbr\u003e Forward: [19, 339, 699]           |\n\nThese models saved as `.safetensors` files.\n\n## Easy-to-run examples\n\n\u003cb\u003eStep 0. Download the models and put them to the *checkpoints* folder\u003c/b\u003e\n\nFor this example, we consider [iCD-SD1.5](https://storage.yandexcloud.net/yandex-research/invertible-cd/iCD-SD1.5_1.tar) using\nreverse: [259, 519, 779, 999], forward: [19, 259, 519, 779] time steps.\n\n\u003cb\u003eStep 1. Load the models \u003c/b\u003e\n\n```Python\nfrom utils.loading import load_models\nfrom diffusers import DDPMScheduler\n\nroot = 'checkpoints'\nldm_stable, reverse_cons_model, forward_cons_model = load_models(\n    model_id=\"runwayml/stable-diffusion-v1-5\",\n    device='cuda',\n    forward_checkpoint=f'{root}/iCD-SD15-forward_19_259_519_779.safetensors',\n    reverse_checkpoint=f'{root}/iCD-SD15-reverse_259_519_779_999.safetensors',\n    r=64,\n    w_embed_dim=512,\n    teacher_checkpoint=f'{root}/sd15_cfg_distill.pt',\n)\n\ntokenizer = ldm_stable.tokenizer\nnoise_scheduler = DDPMScheduler.from_pretrained(\n    \"runwayml/stable-diffusion-v1-5\", subfolder=\"scheduler\", )\n```\n\n\u003cb\u003eStep 2. Specify the configuration according to the downloaded model\u003c/b\u003e\n\n```Python\nfrom utils import p2p, generation\n\nNUM_REVERSE_CONS_STEPS = 4\nREVERSE_TIMESTEPS = [259, 519, 779, 999]\nNUM_FORWARD_CONS_STEPS = 4\nFORWARD_TIMESTEPS = [19, 259, 519, 779]\nNUM_DDIM_STEPS = 50\n\nsolver = generation.Generator(\n    model=ldm_stable,\n    noise_scheduler=noise_scheduler,\n    n_steps=NUM_DDIM_STEPS,\n    forward_cons_model=forward_cons_model,\n    forward_timesteps=FORWARD_TIMESTEPS,\n    reverse_cons_model=reverse_cons_model,\n    reverse_timesteps=REVERSE_TIMESTEPS,\n    num_endpoints=NUM_REVERSE_CONS_STEPS,\n    num_forward_endpoints=NUM_FORWARD_CONS_STEPS,\n    max_forward_timestep_index=49,\n    start_timestep=19)\n\np2p.NUM_DDIM_STEPS = NUM_DDIM_STEPS\np2p.tokenizer = tokenizer\np2p.device = 'cuda'\n```\n\n### Generation with iCD-SD1.5\n\n\u003cb\u003eStep 3. Generate\u003c/b\u003e\n\n```Python\nimport torch\n\nprompt = ['a cute owl with a graduation cap']\ncontroller = p2p.AttentionStore()\n\ngenerator = torch.Generator().manual_seed(150)\ntau = 1.0\nimage, _ = generation.runner(\n    # Playing params\n    guidance_scale=19.0,\n    tau1=tau,  # Dynamic guidance if tau \u003c 1.0\n    tau2=tau,\n\n    # Fixed params\n    is_cons_forward=True,\n    model=reverse_cons_model,\n    w_embed_dim=512,\n    solver=solver,\n    prompt=prompt,\n    controller=controller,\n    generator=generator,\n    latent=None,\n    return_type='image')\n\n# The left image is inversion, the right - editing.\ngeneration.to_pil_images(image).save('test_generation_iCD-SD1.5.jpg')\ngeneration.view_images(image)\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/sd1.5_img.jpg\" width=\"512\"/\u003e\n\u003c/p\u003e\n\n### Editing with iCD-SD1.5\n\n\u003cb\u003eStep 3. Load and invert real image\u003c/b\u003e\n\n```Python\nfrom utils import inversion\n\nimage_path = f\"assets/bird.jpg\"\nprompt = [\"a photo of a bird standing on a branch\"]\n\n(image_gt, image_rec), ddim_latent, uncond_embeddings = inversion.invert(\n    # Playing params\n    image_path=image_path,\n    prompt=prompt,\n\n    # Fixed params\n    is_cons_inversion=True,\n    w_embed_dim=512,\n    inv_guidance_scale=0.0,\n    stop_step=50,\n    solver=solver,\n    seed=10500)\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/bird.jpg\" width=\"512\"/\u003e\n\u003c/p\u003e\n\n\u003cb\u003eStep 4. Edit the image\u003c/b\u003e\n\n```Python\np2p.NUM_DDIM_STEPS = 4\np2p.tokenizer = tokenizer\np2p.device = 'cuda'\n\nprompts = [\"a photo of a bird standing on a branch\",\n           \"a photo of a lego bird standing on a branch\"\n           ]\n\n# Playing params\ncross_replace_steps = {'default_': 0.2, }\nself_replace_steps = 0.2\nblend_word = ((('bird',), ('lego',)))\neq_params = {\"words\": (\"lego\",), \"values\": (3.,)}\n\ncontroller = p2p.make_controller(prompts,\n                                 False, # (is_replacement) True if only one word is changed\n                                 cross_replace_steps,\n                                 self_replace_steps,\n                                 blend_word,\n                                 eq_params)\n\ntau = 0.8\nimage, _ = generation.runner(\n    # Playing params\n    guidance_scale=19.0,\n    tau1=tau,  # Dynamic guidance if tau \u003c 1.0\n    tau2=tau,\n\n    # Fixed params\n    model=reverse_cons_model,\n    is_cons_forward=True,\n    w_embed_dim=512,\n    solver=solver,\n    prompt=prompts,\n    controller=controller,\n    num_inference_steps=50,\n    generator=None,\n    latent=ddim_latent,\n    uncond_embeddings=uncond_embeddings,\n    return_type='image')\n\ngeneration.to_pil_images(image).save('test_editing_iCD-SD1.5.jpg')\ngeneration.view_images(image)\n```\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"assets/lego_bird.jpg\" width=\"512\"/\u003e\n\u003c/p\u003e\n\n\u003e**Note**: \u003cbr\u003e\nPlease note that zero-shot editing is highly sensitive to hyperparameters. Thus, we recommend tuning: cross_replace_steps \n\u003e (from 0.0 to 1.0), self_replace_steps (from 0.0 to 1), tau (0.7 or 0.8 seems to work best), \n\u003e guidance scale (up to 19), and amplify factor (eq_params).\n\nYou can also consider the similar easy-to-run examples for the [SDXL model](running/sdxl)\nor move on to [in-depth examples](running)\n\n## Citation\n\n```bibtex\n@article{starodubcev2024invertible,\n  title={Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps},\n  author={Starodubcev, Nikita and Khoroshikh, Mikhail and Babenko, Artem and Baranchuk, Dmitry},\n  journal={arXiv preprint arXiv:2406.14539},\n  year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyandex-research%2Finvertible-cd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyandex-research%2Finvertible-cd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyandex-research%2Finvertible-cd/lists"}