{"id":18936093,"url":"https://github.com/aim-uofa/oir","last_synced_at":"2025-04-22T13:21:36.897Z","repository":{"id":201009213,"uuid":"706557640","full_name":"aim-uofa/OIR","owner":"aim-uofa","description":"[ICLR 2024] Official PyTorch/Diffusers implementation of \"Object-aware Inversion and Reassembly for Image Editing\"","archived":false,"fork":false,"pushed_at":"2024-08-23T01:06:56.000Z","size":174833,"stargazers_count":84,"open_issues_count":0,"forks_count":2,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-03-27T23:33:07.359Z","etag":null,"topics":["diffusion-models","image-editing","inversion"],"latest_commit_sha":null,"homepage":"https://aim-uofa.github.io/OIR-Diffusion/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aim-uofa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-18T07:26:54.000Z","updated_at":"2025-03-26T04:25:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"1e063779-5b3d-4a26-a075-4beaab512d64","html_url":"https://github.com/aim-uofa/OIR","commit_stats":null,"previous_names":["aim-uofa/oir"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FOIR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FOIR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FOIR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aim-uofa%2FOIR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aim-uofa","download_url":"https://codeload.github.com/aim-uofa/OIR/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250246746,"owners_count":21398919,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diffusion-models","image-editing","inversion"],"created_at":"2024-11-08T12:05:53.546Z","updated_at":"2025-04-22T13:21:36.868Z","avatar_url":"https://github.com/aim-uofa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n\u003c!-- # magic-edit.github.io --\u003e\n\n\u003cp align=\"center\"\u003e\n\n  \u003ch2 align=\"center\"\u003eObject-aware Inversion and Reassembly for Image Editing\u003c/h2\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://zhenyangcs.github.io/\"\u003e\u003cstrong\u003eZhen Yang*\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/dingangui\"\u003e\u003cstrong\u003eGanggui Ding*\u003c/strong\u003e\u003c/a\u003e\n    ·  \n    \u003ca href=\"https://scholar.google.com/citations?user=1ks0R04AAAAJ\u0026hl=zh-CN\"\u003e\u003cstrong\u003eWen Wang*\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://scholar.google.com/citations?user=FaOqRpcAAAAJ\"\u003e\u003cstrong\u003eHao Chen*\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://bohanzhuang.github.io/\"\u003e\u003cstrong\u003eBohan Zhuang†\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://cshen.github.io/\"\u003e\u003cstrong\u003eChunhua Shen*\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr\u003e\n    *Zhejiang University\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;†Monash University\n    \u003cbr\u003e\n    \u003c/br\u003e\n        \u003ca href=\"https://arxiv.org/abs/2310.12149\"\u003e\n        \u003cimg src='https://img.shields.io/badge/Arxiv-OIR-blue' alt='Paper PDF'\u003e\u003c/a\u003e\n        \u003ca href=\"https://aim-uofa.github.io/OIR-Diffusion/\"\u003e\n        \u003cimg src='https://img.shields.io/badge/Project-Website-orange' alt='Project Page'\u003e\u003c/a\u003e\n        \u003ca href=\"https://drive.google.com/file/d/1JX8w0S9PCD9Ipmo9IiICO8R7e1haTGdF/view?usp=sharing\"\u003e\n        \u003cimg src='https://img.shields.io/badge/Dataset-OIR--Bench-green' alt='OIR-Bench'\u003e\u003c/a\u003e\n        \u003ca href=\"https://iclr.cc/virtual/2024/poster/18242\"\u003e\n        \u003cimg src='https://img.shields.io/badge/Video-ICLR-yellow' alt='Video'\u003e\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n\n\u003c!-- \u003cp align=\"center\"\u003e\u003cb\u003eWe will release the code soon!\u003c/b\u003e\u003c/p\u003e --\u003e\n\n## Setup\nThis code was tested with Python 3.9, Pytorch 2.0.1 using pre-trained models through huggingface / diffusers. Specifically, we implemented our method over Stable Diffusion 1.4. Additional required packages are listed in the requirements file. The code was tested on a NVIDIA GeForce RTX 3090 but should work on other cards.\n\n## Getting Started\n1. Download OIR-Bench.\n2. Create the environment and install the dependencies by running:\n```\nconda create -n oir python=3.9\nconda activate oir\npip install -r requirements.txt\n```\n3. Change the **basic_config.py** in **configs/**, change the model path and hyperparameters.\n4. Modify **multi_object_edit.yaml** or **single_object_edit.yaml** in **configs/** according to **multi_object.yaml** and **single_object.yaml** in **OIR-Bench/**.\n5. Run **single_object_edit.py** (Search Metric in paper) or **multi_object_edit.py** (OIR in paper) to implement image editing.\n6. **Option**: Adjust **reassembly_step** and repeat the above process to get better results.\n\n## TODO\n1. Use prompt_change as dict's key may lead to error.\n2. Different editing pairs' masks mustn't have overlap.\n3. Search metric can be an ensemble learning tool. For example, we can use pnp, p2p, OIR ... method to edit an image and we can use search metric to select the optimal editing result.\n4. We can also use the method in TODO 3 to build a high quality dataset to train instruct-based image editing method.\n5. Deploy our method on different foundation model (SDXL, LCM ...)\n\n## Results\n\n### OIR results\n\u003cp align=\"center\"\u003e\n  \u003ctable align=\"center\"\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"./assets/OIR_result_1.png\"\u003e\u003c/img\u003e\n      \u003c!-- \u003cimg src=\"./assets/OIR_result_2.png\"\u003e\u003c/img\u003e\n      \u003cimg src=\"./assets/OIR_result_3.png\"\u003e\u003c/img\u003e --\u003e\n    \u003c/td\u003e\n  \u003c/table\u003e\n\u003c/p\u003e\n\n### Visualization of the search metric\n\u003cp align=\"center\"\u003e\n  \u003ctable align=\"center\"\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"./assets/search_metric_1.png\"\u003e\u003c/img\u003e\n      \u003c!-- \u003cimg src=\"./assets/search_metric_2.png\"\u003e\u003c/img\u003e --\u003e\n    \u003c/td\u003e\n  \u003c/table\u003e\n\u003c/p\u003e\n\n\n\n## Acknowlegment\nMany thanks for the generous help in building the project website from Minghan Li.\n\n## License\nFor non-commercial academic use, this project is licensed under [the 2-clause BSD License](https://opensource.org/license/bsd-2-clause). \nFor commercial use, please contact [Chunhua Shen](mailto:chhshen@gmail.com).\n\n\n\n\n## BibTeX\n```BibTeX\n@inproceedings{yang2024objectaware,\ntitle     = {Object-Aware Inversion and Reassembly for Image Editing},\nauthor    = {Zhen Yang and Ganggui Ding and Wen Wang and Hao Chen and Bohan Zhuang and Chunhua Shen},\nbooktitle = {The Twelfth International Conference on Learning Representations},\nyear      = {2024},\nurl       = {https://openreview.net/forum?id=dpcVXiMlcv}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faim-uofa%2Foir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faim-uofa%2Foir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faim-uofa%2Foir/lists"}