{"id":13487863,"url":"https://github.com/garibida/ReNoise-Inversion","last_synced_at":"2025-03-27T23:31:53.664Z","repository":{"id":228830315,"uuid":"773127072","full_name":"garibida/ReNoise-Inversion","owner":"garibida","description":"Officail Implementation for \"ReNoise: Real Image Inversion Through Iterative Noising\"","archived":false,"fork":false,"pushed_at":"2024-07-03T20:01:51.000Z","size":9132,"stargazers_count":191,"open_issues_count":7,"forks_count":8,"subscribers_count":15,"default_branch":"main","last_synced_at":"2024-10-30T23:35:57.293Z","etag":null,"topics":["diffusion-models","image-editing","inversion","lcm","lcm-lora","sdxl-turbo","stable-diffusion","text-to-image"],"latest_commit_sha":null,"homepage":"https://garibida.github.io/ReNoise-Inversion/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/garibida.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-16T20:27:51.000Z","updated_at":"2024-10-30T15:17:55.000Z","dependencies_parsed_at":"2024-04-21T17:30:54.228Z","dependency_job_id":"3c36b331-66be-49c3-839d-53ea92c53986","html_url":"https://github.com/garibida/ReNoise-Inversion","commit_stats":null,"previous_names":["garibida/renoise-inversion"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/garibida%2FReNoise-Inversion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/garibida%2FReNoise-Inversion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/garibida%2FReNoise-Inversion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/garibida%2FReNoise-Inversion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/garibida","download_url":"https://codeload.github.com/garibida/ReNoise-Inversion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245944020,"owners_count":20697945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion-models","image-editing","inversion","lcm","lcm-lora","sdxl-turbo","stable-diffusion","text-to-image"],"created_at":"2024-07-31T18:01:05.320Z","updated_at":"2025-03-27T23:31:49.321Z","avatar_url":"https://github.com/garibida.png","language":"Python","funding_links":[],"categories":["Diffusion Models Inversion"],"sub_categories":["Train-Free"],"readme":"# ReNoise: Real Image Inversion Through Iterative Noising\n\u003e **Daniel Garibi, Or Patashnik, Andrey Voynov, Hadar Averbuch-Elor, Daniel Cohen-Or**\n\u003e\n\u003e Recent advancements in text-guided diffusion models have unlocked powerful image manipulation capabilities. However, applying these methods to real images necessitates the inversion of the images into the domain of the pretrained diffusion model. Achieving faithful inversion remains a challenge, particularly for more recent models trained to generate images with a small number of denoising steps. In this work, we introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. Building on reversing the diffusion sampling process, our method employs an iterative renoising mechanism at each inversion sampling step. This mechanism refines the approximation of a predicted point along the forward diffusion trajectory, by iteratively applying the pretrained diffusion model, and averaging these predictions. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models. Through comprehensive evaluations and comparisons, we show its effectiveness in terms of both accuracy and speed. Furthermore, we confirm that our method preserves editability by demonstrating text-driven image editing on real images.\n\n\u003ca href=\"https://garibida.github.io/ReNoise-Inversion/\"\u003e\u003cimg src=\"https://img.shields.io/static/v1?label=Project\u0026message=Website\u0026color=red\" height=20.5\u003e\u003c/a\u003e \n\u003ca href=\"https://arxiv.org/abs/2403.14602\"\u003e\u003cimg src=\"https://img.shields.io/badge/arXiv-ReNoise-b31b1b.svg\" height=20.5\u003e\u003c/a\u003e\n[![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/garibida/ReNoise-Inversion)\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"docs/teaser.jpg\" width=\"800px\"/\u003e\n\u003c/p\u003e\n\n## Description\nOfficial implementation of our ReNoise paper.\n\n## Environment Setup\nOur code builds on the requirement of the `diffusers` library. To set up the environment, please run:\n```\nconda env create -f environment.yaml\nconda activate renoise_inversion\n```\nor install requirements:\n```\npip install -r requirements.txt\n```\n## Demo\nTo run a local demo of the project, run the following:\n```\ngradio gradio_app.py\n```\n\n## Usage\nThere are three examples of how to use the inversion in Stable Diffusion, SDXL, and SDXL Turbo. You can find these examples in the `examples/` directory.\n\n### Inversion\n\nWe have created a diffusers pipe to perform the inversion. You can use the following code to use ReNoise in your project:\n\n```\nfrom src.eunms import Model_Type, Scheduler_Type\nfrom src.utils.enums_utils import get_pipes\nfrom src.config import RunConfig\n\nfrom main import run as invert\n\nmodel_type = Model_Type.SDXL_Turbo\nscheduler_type = Scheduler_Type.EULER\npipe_inversion, pipe_inference = get_pipes(model_type, scheduler_type, device=device)\n\ninput_image = Image.open(\"example_images/lion.jpeg\").convert(\"RGB\").resize((512, 512))\nprompt = \"a lion in the field\"\n\nconfig = RunConfig(model_type = model_type,\n                    scheduler_type = scheduler_type)\n\nrec_img, inv_latent, noise, all_latents = invert(input_image,\n                                                 prompt,\n                                                 config,\n                                                 pipe_inversion=pipe_inversion,\n                                                 pipe_inference=pipe_inference,\n                                                 do_reconstruction=False)\n```\n\nYou can controll the inversion paremeters using the following attributes in the `RunConfig`:\n- `num_inference_steps` - Number of denoise steps.\n- `num_inversion_steps` - Number of inversion steps.\n- `guidance_scale` - Guidence scale during the inversion.\n- `num_renoise_steps` - Number of ReNoise steps.\n- `max_num_renoise_steps_first_step` - Max number of ReNoise steps when `T\u003c250`\n- `inversion_max_step` - Inversion strength. The number of denoising steps depends on the amount of noise initially added. When strength is `1.0`, the image will be inverted to complete noise and the denoising process will run for the full number of steps. When strength is `0.5`, the image will be inverted to half noise and the denoising process will run for half of the steps.\n- `num_inference_steps` - Number of denoise steps\n- `average_latent_estimations` - Perform estimations averaging.\n- `average_first_step_range` - Averaging range when `T\u003c250`. The value is tuple, for example `(0, 5)`.\n- `average_first_step_range` - Averaging range when `T\u003e250`. The value is tuple, for example `(8, 10)`.\n- `noise_regularization_lambda_ac` - Noise regularization pairwise lambda.\n- `noise_regularization_lambda_kl` - Noise regularization patch KL divergence lambda.\n- `perform_noise_correction` - Perform noise correction.\n\nIn case of stochastic sampler add the following to use the same \\epsilon_t as in the inversion process.\n```\npipe_inference.scheduler.set_noise_list(noise)\n```\n\n### Edit\nTo edit images using ReNoise you can use the following code:\n```\nfrom src.eunms import Model_Type, Scheduler_Type\nfrom src.utils.enums_utils import get_pipes\nfrom src.config import RunConfig\n\nfrom main import run as invert\n\nmodel_type = Model_Type.SDXL_Turbo\nscheduler_type = Scheduler_Type.EULER\npipe_inversion, pipe_inference = get_pipes(model_type, scheduler_type, device=device)\n\ninput_image = Image.open(\"example_images/lion.jpeg\").convert(\"RGB\").resize((512, 512))\nprompt = \"a lion in the field\"\n\nconfig = RunConfig(model_type = model_type,\n                    scheduler_type = scheduler_type)\n\nedit_img, inv_latent, noise, all_latents = invert(input_image,\n                                                 prompt,\n                                                 config,\n                                                 pipe_inversion=pipe_inversion,\n                                                 pipe_inference=pipe_inference,\n                                                 do_reconstruction=True,\n                                                 edit_prompt=\"a tiger in the field\"\n                                                 )\n\nedit_img.save(\"result.png\")\n```\n\n## Acknowledgements \nThis code builds on the code from the [diffusers](https://github.com/huggingface/diffusers) library. In addition, we \nborrow code from the following repositories: \n- [Pix2PixZero](https://github.com/pix2pixzero/pix2pix-zero) for noise regularization.\n- [sdxl_inversions](https://github.com/cloneofsimo/sdxl_inversions) for initial implementation of DDIM inversion in SDXL.\n\n\n## Citation\nIf you use this code for your research, please cite the following work: \n```\n@misc{garibi2024renoise,\n      title={ReNoise: Real Image Inversion Through Iterative Noising}, \n      author={Daniel Garibi and Or Patashnik and Andrey Voynov and Hadar Averbuch-Elor and Daniel Cohen-Or},\n      year={2024},\n      eprint={2403.14602},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgaribida%2FReNoise-Inversion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgaribida%2FReNoise-Inversion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgaribida%2FReNoise-Inversion/lists"}