{"id":13487576,"url":"https://github.com/tirer-lab/DDPG","last_synced_at":"2025-03-27T22:31:39.185Z","repository":{"id":214101586,"uuid":"733681628","full_name":"tirer-lab/DDPG","owner":"tirer-lab","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-02T15:14:45.000Z","size":14418,"stargazers_count":49,"open_issues_count":0,"forks_count":6,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-02T16:24:40.638Z","etag":null,"topics":["cvpr","cvpr2024","deblurring","diffusion-models","inverse-problems","score-based","super-resolution","zero-shot"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tirer-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-19T22:11:11.000Z","updated_at":"2025-03-02T15:14:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"c6901d92-ace5-461d-96b9-e019250f6bd0","html_url":"https://github.com/tirer-lab/DDPG","commit_stats":null,"previous_names":["tirer-lab/ddpg"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FDDPG","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FDDPG/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FDDPG/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FDDPG/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tirer-lab","download_url":"https://codeload.github.com/tirer-lab/DDPG/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245935628,"owners_count":20696672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cvpr","cvpr2024","deblurring","diffusion-models","inverse-problems","score-based","super-resolution","zero-shot"],"created_at":"2024-07-31T18:01:00.786Z","updated_at":"2025-03-27T22:31:39.168Z","avatar_url":"https://github.com/tirer-lab.png","language":"Python","funding_links":[],"categories":["VAR model"],"sub_categories":[],"readme":"# Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance\n\n## 📖[**Paper**](https://arxiv.org/pdf/2312.16519.pdf) (CVPR 2024)\n\n[Tomer Garber](https://scholar.google.com/citations?user=-ZciND8AAAAJ), [Tom Tirer](https://scholar.google.com/citations?user=_6bZV20AAAAJ)\n\nThe Open University of Israel and Bar-Ilan University\n\n# Introduction\n\nThis repository contains the code release for *Image Restoration by Denoising Diffusion Models with Iteratively\nPreconditioned Guidance* (***DDPG***).\n\nMain idea: identifying [back-projection](https://arxiv.org/abs/1710.06647) (BP) guidance (used recently under the names \"pseudoinverse\" or \"range/null-space\" guidance) as a preconditioned version of least squares (LS) guidance, and accordingly, devising a guidance technique with iteration-dependent preconditioning that traverses from BP to LS, enjoying the benefits of both.\n\n## Supported degradations\n\n1. Super-Resolution (Bicubic)\n2. Gaussian Deblurring\n3. Motion Deblurring\n\nThe code can be very easily adapted to super-resolution and deblurring with any kernel.  \nExtension to other linear measurement models is also possible.\n\n# Setup\n\n## Installation\n\n### Clone this repository\n\nTo clone this repository and the code, run:\n\n```bash\ngit clone https://github.com/tirer-lab/DDPG.git\n```\n\n### Environment\n\nThere are several dependencies required, and you may install it via *pip* or *docker*.\nThe code was written and tested on Python 3.8 and PyTorch 1.9.0.\n\n#### Pip\n\n```\npip install torch torchvision lpips numpy tqdm pillow pyYaml pandas scipy\n```\n\nInstall the relevant torch and torchvision versions according to your setup, for example:\n\n```\npip install torch==1.9.0+cu111 torchvision==0.10.1+cu111 -f https://download.pytorch.org/whl/torch_stable.html\n```\n\n#### Docker\n\nThe repository contains [Dockerfile](Dockerfile), in order to use it run (after cloning this repo and `cd` to it):\n\n```bash\ndocker build .\n```\n\nIf you wish to run *IDPG* instead of *DDPG*, You can swap the [`CMD`](Dockerfile#L11) commands in the Docker file.\n\n## Pre-Trained Models\n\nTo download the models used in the paper:\n\n### CelebA-HQ\n\nThe CelebA-HQ model checkpoint can be\nfound [here](https://drive.google.com/file/d/1wSoA5fm_d6JBZk4RZ1SzWLMgev4WqH21/view?usp=share_link).\nDownload it and place it in `DDPG/exp/logs/celeba/`.\n\n### ImageNet\nThe ImageNet model checkpoint can be\nfound [here](https://openaipublic.blob.core.windows.net/diffusion/jul-2021/256x256_diffusion_uncond.pt).\nDownload it and place it in `DDPG/exp/logs/imagenet/`.\n\n## Quick Start\nRun the following commands to get immediate DDPG results:\n\n1. CelebA noiseless SRx4:\n    ```bash\n    python main.py --config celeba_hq.yml --path_y celeba_hq --deg sr_bicubic --sigma_y 0 \\\n    -i DDPG_celeba_sr_bicubic_sigma_y_0 --inject_noise 1 --zeta 0.7 --step_size_mode 0 \\\n    --deg_scale 4 --operator_imp SVD\n    ```\n\n2. CelebA Gaussian deblurring with sigma_y=0.05:\n    ```bash\n    python main.py --config celeba_hq.yml --path_y celeba_hq --deg deblur_gauss --sigma_y 0.05 \\\n    -i DDPG_celeba_deblur_gauss_sigma_y_0.05 --inject_noise 1 --gamma 8 --zeta 0.5 --eta_tilde 0.7 \\\n    --step_size_mode 1 --operator_imp FFT\n    ```\n\nThe results will be in `DDPG/exp/image_samples/`.\n## Full Datasets\n\nThe datasets used in the paper are CelebA-HQ and ImageNet. Both can be found in: \n[[Google drive](https://drive.google.com/drive/folders/1cSCTaBtnL7OIKXT4SVME88Vtk4uDd_u4?usp=sharing)] [[Baidu drive](https://pan.baidu.com/s/1tQaWBqIhE671v3rrB-Z2mQ?pwd=twq0)].\n\nAfter you download the datasets, place each dataset in the relevant directory:\n\n1. CelebA-HQ - Place the dataset in `DDPG/exp/datasets/celeba/`.\n2. ImageNet -  Place the dataset in `DDPG/exp/datasets/imagenet/`.\n   1. Download the file `imagenet_val_1k.txt` from the links above as well, and place it in `DDPG/exp`. Rename this file to `imagenet_val.txt` in order for the code to use it.\n\n## Motion Deblur\n\nFor motion deblur we used the following git repository to generate the\nkernels: https://github.com/LeviBorodenko/motionblur.\n\nClone that repository and copy the *motionblur.py* file into `DDPG/functions`.\n\nAs mentioned in the paper, we used motion deblur kernels with `intensity=0.5`.\n\n## Parameters\n\nThe general python command to run the code is:\n```\npython main.py --config {config}.yml --path_y {dataset_folder} --deg {deg} --sigma_y {sigma_y}\n-i {image_folder} --inject_noise {inject_noise} --gamma {gamma} --zeta {zeta} --eta_tilde {eta_tilde}\n--step_size_mode {step_size_mode} --operator_imp {operator_implementation} --save_y {save_observation}\n--scale_ls {scale_for_gLS}\n```\n\nWhere:\n\n- `config`: The name of the yml to use to configure the model used.\n- `dataset_folder`: The name of the directory containing the image dataset.\n- `deg`: the degradation type to use. Used in paper: `sr_bicubic`, `deblur_gauss`, `motion_deblur`\n    - When using `sr_bicubic`, the flag `--deg_scale 4` is also required\n- `sigma_y`: Noise level. Noise levels used in paper: `0, 0.01, 0.05, 0.1`.\n- `image_folder`: Name of directory for output images.\n- `inject_noise`: Whether to inject noise (1) and run *DDPG* or not (0) and run *IDPG*.\n- `gamma`: The Gamma hyperparameter used in the paper.\n- `zeta`: The Zeta hyperparameter used in the paper.\n- `eta_tilde`: The Eta hyperparameter used in the paper.\n- `step_size_mode`: Which step size mode to use. In the paper, `step_size_mode=0` (fixed 1) was used for IDPG, noiseless DDPG and \nDDPG with noise level `0.01`. `step_size_mode=1` (certain decay) was used for the rest of the DDPG runs.\n- `operator_implementation` - Whether to use `SVD` or `FFT`. Defaults to `FFT`.\n- `scale_ls` - The `c` hyperparameter used in the paper, which is Least Squares guidance scale. Defaults to `1`. \n- `save_observation` - Whether to save the observed image (`y`) or not. Defaults to `False`.\n\n\nAdditionally, you can configure the sampling steps (defaults to `100` in the paper). In each yml config under `configs` directory\n(`celeba_hq.yml`, `imagenet_256.yml` and `imagenet_256_cc.yml`) you may change:\n```yaml\nsampling:\n  T_sampling: \u003cdesired_sampling_steps\u003e\n```\n\n## Evaluation\n\nIn order to reproduce the paper's results, there are 2 evaluation scripts:\n\n1. [evaluation_DDPG.sh](evaluation_DDPG.sh) for *DDPG* results.\n2. [evaluation_IDPG.sh](evaluation_IDPG.sh) for *IDPG* results.\n\nBoth scripts contain all tasks mentioned in the paper with the relevant configuration.\n\n## Qualitative Results\n\n![Qualitative Results](figs/qualitative_results.png)\n\nAdditional results can be found in the paper, including PSNR and LPIPS results compared to competitors.\n\n## Citations\nIf you used this repository in your research, please cite the paper:\n```\n@inproceedings{garber2023image,\n  title={Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance},\n  author={Garber, Tomer and Tirer, Tom},\n  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},\n  year={2024}\n}\n```\n\nThis implementation is inspired by https://github.com/bahjat-kawar/ddrm and https://github.com/wyhuai/DDNM.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftirer-lab%2FDDPG","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftirer-lab%2FDDPG","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftirer-lab%2FDDPG/lists"}