{"id":13487582,"url":"https://github.com/Zj-BinXia/DiffIR","last_synced_at":"2025-03-27T22:31:43.463Z","repository":{"id":182812804,"uuid":"668640050","full_name":"Zj-BinXia/DiffIR","owner":"Zj-BinXia","description":"This project is the official implementation of 'Diffir: Efficient diffusion model for image restoration', ICCV2023","archived":false,"fork":false,"pushed_at":"2024-04-01T11:24:47.000Z","size":14885,"stargazers_count":349,"open_issues_count":10,"forks_count":15,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-04-01T12:36:49.730Z","etag":null,"topics":["deblurring","diffusion-model","inpainting","super-resolution"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Zj-BinXia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-07-20T09:19:57.000Z","updated_at":"2024-04-15T06:23:59.227Z","dependencies_parsed_at":"2023-09-30T17:29:50.895Z","dependency_job_id":"e3ac2852-b76d-4790-ad9b-a726c3133ec3","html_url":"https://github.com/Zj-BinXia/DiffIR","commit_stats":null,"previous_names":["zj-binxia/diffir"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zj-BinXia%2FDiffIR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zj-BinXia%2FDiffIR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zj-BinXia%2FDiffIR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zj-BinXia%2FDiffIR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Zj-BinXia","download_url":"https://codeload.github.com/Zj-BinXia/DiffIR/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222322034,"owners_count":16966433,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deblurring","diffusion-model","inpainting","super-resolution"],"created_at":"2024-07-31T18:01:00.851Z","updated_at":"2025-03-27T22:31:43.453Z","avatar_url":"https://github.com/Zj-BinXia.png","language":"Jupyter Notebook","funding_links":[],"categories":["VAR model"],"sub_categories":[],"readme":"# DiffIR: Efficient diffusion model for image restoration (ICCV2023)\n\n[Paper](https://arxiv.org/pdf/2303.09472.pdf) | [Project Page](https://github.com/Zj-BinXia/DiffIR) | [pretrained models](https://drive.google.com/drive/folders/10miVILiopE414GyaSZM3EFAZITeY9q0p?usp=sharing)\n\n#### News\n- **Dec 19, 2023:**  We propose reference-based DiffIR (DiffRIR) to alleviate texture, brightness, and contrast disparities between generated and preserved regions during image editing, such as inpainting and outpainting. All training and inference codes and pre-trained models (x1, x2, x4) are released at [Github](https://github.com/Zj-BinXia/DiffRIR)\n- **Sep 10, 2023:** For real-world SR, we release x1 and x2 pre-trained models.\n- **Sep 6, 2023:** For real-world SR and SRGAN, we can test [LR images without GT images](DiffIR-RealSR/options/test_DiffIRS2_GAN_x4.yml) and [inference](DiffIR-RealSR/inference_diffir.py). \n- **August 31, 2023:** For real-world SR and SRGAN tasks, we updated 2x SR training files. \n- **August 28, 2023:** For real-world SR tasks, we released the pretrained models [RealworldSR-DiffIRS2-GANV2](https://drive.google.com/drive/folders/1H4DU-9fB15fSz-OFko00HlWYbNSqmAKq?usp=sharing) and [training files](DiffIR-RealSR/options/train_DiffIRS2_GAN_x4_V2.yml) that are more focused on perception rather than the distortion， which can be used to super-resolve AIGC generated images. \n- **July 20, 2023:** Training\u0026Testing codes and pre-trained models are released!\n\n\u003chr /\u003e\n\n\n\u003e **Abstract:** *Diffusion model (DM) has achieved SOTA performance by modeling the image synthesis process into a sequential application of a denoising network. However, different from image synthesis, image restoration (IR) has a strong constraint to generate results in accordance with ground-truth. Thus, for IR, traditional DMs running massive iterations on a large model to estimate whole images or feature maps is inefficient. To address this issue, we propose an efficient DM for IR (DiffIR), which consists of a compact IR prior extraction network (CPEN), dynamic IR transformer (DIRformer), and denoising network. Specifically, DiffIR has two training stages: pretraining and training DM. In pretraining, we input ground-truth images into CPEN$_{S1}$ to capture a compact IR prior representation (IPR) to guide DIRformer. In the second stage, we train the DM to directly estimate the same IRP as pretrained CPEN$_{S1}$ only using LQ images. We observe that since the IPR is only a compact vector,  DiffIR can use fewer iterations than traditional DM to obtain accurate estimations and generate more stable and realistic results. Since the iterations are few, our DiffIR can adopt a joint optimization of CPEN$_{S2}$, DIRformer, and denoising network, which can further reduce the estimation error influence. We conduct extensive experiments on several IR tasks and achieve SOTA performance while consuming less computational costs.* \n\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"800\" src=\"figs/method.jpg\"\u003e\n\u003c/p\u003e\n\n---\n\n## Installation\n\nFor inpainting, see [pip.sh](DiffIR-inpainting/pip.sh) for the installation of dependencies required to run DiffIR.\n\nFor GAN based single-image super-resolution, see [pip.sh](DiffIR-SRGAN/pip.sh) for the installation of dependencies required to run DiffIR.\n\nFor real-world super-resolution, see [pip.sh](DiffIR-RealSR/pip.sh) for the installation of dependencies required to run DiffIR.\n\nFor motion deblurring, see [pip.sh](DiffIR-demotionblur/pip.sh) for the installation of dependencies required to run DiffIR.\n\n\n\n## Training and Evaluation\n\nTraining and Testing instructions for Inpainting, GAN based single-image super-resolution, real-world super-resolution, and motion deblurring are provided in their respective directories. Here is a summary table containing hyperlinks for easy navigation:\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003cth align=\"left\"\u003eTask\u003c/th\u003e\n    \u003cth align=\"center\"\u003eTraining Instructions\u003c/th\u003e\n    \u003cth align=\"center\"\u003eTesting Instructions\u003c/th\u003e\n    \u003cth align=\"center\"\u003eDiffIR's Pretrained Models\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"left\"\u003eInpainting\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-inpainting/README.md#training\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-inpainting/README.md#evaluation\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://drive.google.com/drive/folders/1RQXRWMqVaAsyyQt8T-3KtpS68ef8dh90?usp=drive_link\"\u003eDownload\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eGAN based single-image super-resolution\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-SRGAN/README.md#training\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-SRGAN/README.md#evaluation\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://drive.google.com/drive/folders/1Mmhz6Sx9tz-n3QJAd6w-UlxdugTEH2fV?usp=drive_link\"\u003eDownload\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eReal-world super-resolution\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-RealSR/README.md#training\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-RealSR/README.md#evaluation\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://drive.google.com/drive/folders/1G3Ep0xd-uBpIXGZFdWzH1uVCOpJaqkOF?usp=drive_link\"\u003eDownload\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eMotion deblurring\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-demotionblur/README.md#training\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"DiffIR-demotionblur/README.md#evaluation\"\u003eLink\u003c/a\u003e\u003c/td\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"https://drive.google.com/drive/folders/1JWYaP9VVPX_Mh2w1Vezn74hck-oWSyMh?usp=drive_link\"\u003eDownload\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Results\nExperiments are performed for different image processing tasks including, inpainting, GAN-based single-image super-resolution, real-world super-resolution, and motion deblurring. \n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eInpainting\u003c/strong\u003e (click to expand) \u003c/summary\u003e\n\u003cimg src = \"figs/inpainting-quan.jpg\"\u003e \n\u003cimg src = \"figs/inpainting-qual.jpg\"\u003e \n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eGAN-based single-image super-resolution\u003c/strong\u003e (click to expand) \u003c/summary\u003e\n\u003cimg src = \"figs/SISR-quan.jpg\"\u003e  \n\u003cimg src = \"figs/SISR-qual.jpg\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eReal-world super-resolution\u003c/strong\u003e (click to expand) \u003c/summary\u003e\n  \n\u003cimg src = \"figs/realworldsr-quan.jpg\"\u003e\n\u003cimg src = \"figs/realworldsr-qual.jpg\"\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eMotion deblurring\u003c/strong\u003e (click to expand) \u003c/summary\u003e\n  \n\u003cimg src = \"figs/deblur-quan.jpg\"\u003e\n\u003cimg src = \"figs/deblur-qual.jpg\"\u003e\n\u003c/details\u003e\n\n## Citation\nIf you use DiffIR, please consider citing:\n\n    @article{xia2023diffir,\n      title={Diffir: Efficient diffusion model for image restoration},\n      author={Xia, Bin and Zhang, Yulun and Wang, Shiyin and Wang, Yitong and Wu, Xinglong and Tian, Yapeng and Yang, Wenming and Van Gool, Luc},\n      journal={ICCV},\n      year={2023}\n    }\n\n\n## Contact\nShould you have any question, please contact zjbinxia@gmail.com\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZj-BinXia%2FDiffIR","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FZj-BinXia%2FDiffIR","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZj-BinXia%2FDiffIR/lists"}