{"id":23916265,"url":"https://github.com/tirer-lab/CM4IR","last_synced_at":"2025-09-11T01:30:52.899Z","repository":{"id":271083860,"uuid":"909028562","full_name":"tirer-lab/CM4IR","owner":"tirer-lab","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-02T19:37:45.000Z","size":3302,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-02T19:39:17.230Z","etag":null,"topics":["consistency-models","cvpr","cvpr2025","deblurring","diffusion-models","inpainting","inverse-problems","score-based","super-resolution","zero-shot"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tirer-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-27T15:17:10.000Z","updated_at":"2025-04-02T19:29:10.000Z","dependencies_parsed_at":"2025-01-05T12:01:56.956Z","dependency_job_id":"49f9f1b5-2e62-45d9-9fab-6cc0105e2080","html_url":"https://github.com/tirer-lab/CM4IR","commit_stats":null,"previous_names":["tirer-lab/cm4ir"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tirer-lab/CM4IR","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FCM4IR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FCM4IR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FCM4IR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FCM4IR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tirer-lab","download_url":"https://codeload.github.com/tirer-lab/CM4IR/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tirer-lab%2FCM4IR/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274559831,"owners_count":25307988,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-10T02:00:12.551Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consistency-models","cvpr","cvpr2025","deblurring","diffusion-models","inpainting","inverse-problems","score-based","super-resolution","zero-shot"],"created_at":"2025-01-05T12:00:57.064Z","updated_at":"2025-09-11T01:30:52.886Z","avatar_url":"https://github.com/tirer-lab.png","language":"Python","funding_links":[],"categories":["VAR model"],"sub_categories":[],"readme":"# Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond)\n\n## 📖[**Paper**](https://arxiv.org/abs/2412.20596.pdf) (CVPR 2025)\n\n[Tomer Garber](https://scholar.google.com/citations?user=-ZciND8AAAAJ), [Tom Tirer](https://scholar.google.com/citations?user=_6bZV20AAAAJ)\n\nBar-Ilan University\n\n# Introduction\n\nThis repository contains the code release for *Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond)* [***CM4IR***].\n\n## Supported degradations\n\n1. Super-Resolution (Bicubic)\n2. Gaussian Deblurring\n3. Inpainting\n\n# Setup\n\n## Installation\n\n### Clone this repository\n\nTo clone this repository and the code, run:\n\n```bash\ngit clone https://github.com/tirer-lab/CM4IR.git\n```\n\n### Environment\n\nThere are several dependencies required, and you may install it via *pip* or *Docker*.\nThe code was written and tested on Python 3.8 and PyTorch 1.9.0.\n\n#### Pip\n\n```bash\npip install -r requirements.txt\n```\n\nFor hardware specific packages, such as torch, install the relevant versions according to your setup, for example:\n\n```bash\npip install torch==1.9.0+cu111 torchvision==0.10.1+cu111 -f https://download.pytorch.org/whl/torch_stable.html\n```\n\n#### Docker\n\nThe repository contains [Dockerfile](Dockerfile), in order to use it run (after cloning this repo and `cd` to it):\n\n```bash\ndocker build .\n```\n\n## Pre-Trained Models\nWe used the official pre-trained Consistency Models provided by OpenAI in the [CM repo](https://github.com/openai/consistency_models?tab=readme-ov-file#pre-trained-models).\n\nTo set the models used in the paper:\n\n### LSUN Bedroom\nThe LSUN Bedroom 256x256 model checkpoint can be\nfound [here](https://openaipublic.blob.core.windows.net/consistency/cd_bedroom256_lpips.pt).\nDownload it and place it in `CM4IR/exp/logs/lsun_bedroom/`.\n\n### LSUN Cat\nThe LSUN Cat 256x256 model checkpoint can be\nfound [here](https://openaipublic.blob.core.windows.net/consistency/cd_cat256_lpips.pt).\nDownload it and place it in `CM4IR/exp/logs/lsun_cat/`.\n\n### ImageNet\nThe ImageNet 64x64 model checkpoint can be\nfound [here](https://openaipublic.blob.core.windows.net/consistency/cd_imagenet64_lpips.pt).\nDownload it and place it in `CM4IR/exp/logs/imagenet/`.\n\n## Quick Start\nRun the following commands to get immediate CM4IR results:\n\n1. LSUN Cat SRx4 with sigma_y=0.05:\n    ```bash\n    python main.py --config lsun_cat_256.yml --path_y lsun_cat --deg sr_bicubic --deg_scale 4 \\\n   --sigma_y 0.05 -i CM4IR_lsun_cat_sr_bicubic_sigma_y_0.05 --iN 250 --gamma 0.2 \\\n   --model_ckpt lsun_cat/cd_cat256_lpips.pt\n    ```\n\n2. LSUN Bedroom Gaussian deblurring with sigma_y=0.025:\n    ```bash\n    python main.py --config lsun_bedroom_256.yml --path_y lsun_bedroom --deg deblur_gauss --sigma_y 0.025 \\\n    -i CM4IR_lsun_bedroom_deblur_gauss_sigma_y_0.025 --iN 75 --gamma 0.01 --zeta 3 \\\n   --model_ckpt lsun_bedroom/cd_bedroom256_lpips.pt\n    ```\n3. ImageNet 64 SRx2 with sigma_y=0.01:\n    ```bash\n    python main.py --config imagenet_64_cc.yml --path_y imagenet --deg sr_bicubic --deg_scale 2 \\\n   --sigma_y 0.01 -i CM4IR_imagenet_sr_bicubic_sigma_y_0.01 --iN 50 --gamma 0.01 \\\n   --model_ckpt imagenet/cd_imagenet64_lpips.pt\n    ```\n\nThe results will be in `CM4IR/exp/image_samples/`.\n## Full Datasets\n\nThe datasets used in the paper are LSUN bedroom, LSUN cat and ImageNet.\n### LSUN\nBoth LSUN Bedroom and LSUN Cat validation sets used in the paper can be found [here](https://drive.google.com/drive/folders/1umSbW_91LTJuK11Il_pmleC4OPei7LAE?usp=sharing).\nAdditionally, the datasets can be downloaded from source:\n\n#### LSUN Bedroom\nThe LSUN Bedroom dataset can be downloaded using the official download code in [this repository](https://github.com/fyu/lsun).\n\n#### LSUN Cat\nThe LSUN Cat dataset can't be downloaded with the script mentioned above, but it can be found [here](http://dl.yf.io/lsun/objects/).\nWe used the last 300 images as validation set.\n\n### ImageNet\nImageNet can be found in: \n[[Google drive](https://drive.google.com/drive/folders/1cSCTaBtnL7OIKXT4SVME88Vtk4uDd_u4?usp=sharing)] [[Baidu drive](https://pan.baidu.com/s/1tQaWBqIhE671v3rrB-Z2mQ?pwd=twq0)].\n\n---\nAfter you download the datasets, place each dataset in the relevant directory:\n\n1. LSUN Bedroom - Place the dataset in `CM4IR/exp/datasets/lsun_bedroom`\n2. LSUN Cat - Place the dataset in `CM4IR/exp/datasets/lsun_cat`\n3. ImageNet - Place the dataset in `CM4IR/exp/datasets/imagenet/`.\n   1. Download the file `imagenet_val_1k.txt` from the links above as well, and place it in `CM4IR/exp`. Rename this file to `imagenet_val.txt` in order for the code to use it.\n\n\n## Parameters\n\nThe general python command to run the code is:\n```\npython main.py --config {config}.yml --path_y {dataset_folder} --deg {deg} --sigma_y {sigma_y}\n-i {image_folder} --iN {iN} --gamma {gamma} --zeta {zeta} --eta {eta} --operator_imp {operator_implementation} \n--deltas {deltas} --save_y {save_observation}\n```\n\nWhere:\n\n- `config`: The name of the yml to use to configure the model used.\n- `model_ckpt`: Name of the model ckpt file in exp/logs. The model should be in `exp/logs/\u003cmodel_ckpt\u003e`. For example: `lsun_bedroom/cd_bedroom256_lpips.pt`.\n- `dataset_folder`: The name of the directory containing the image dataset.\n- `deg`: the degradation type to use. Used in paper: `sr_bicubic`, `deblur_gauss`, `inpainting`\n    - When using `sr_bicubic`, the flag `--deg_scale` is also required\n    - When using `inpainting`, the flag `--inpainting_mask_path` is also required\n- `sigma_y`: Noise level. Noise levels used in paper: `0, 0.01, 0.025, 0.05`.\n- `image_folder`: Name of directory for output images.\n- `iN`: The iN hyperparameter used in the paper.\n- `gamma`: The Gamma hyperparameter used in the paper.\n- `zeta`: The Zeta hyperparameter used in the paper.\n- `eta`: The Eta hyperparameter used in the paper.\n- `deltas`: The deltas hyperparameters used the in the paper. Should be a string of comma seperated values, for example: \"0.1,0.2,0.3,0.4\".\n- `deltas_injection_type`: `0` to inject deltas to the boundary scaling as well, `1` to inject only to the model input.\n- `operator_implementation` - Whether to use `SVD` or `FFT` operator implementations. Defaults to `SVD`.\n- `save_observation` - Whether to save the observed image (`y`) or not. Defaults to `False`.\n\n\nAdditionally, you can configure the sampling steps (defaults to `4` in the paper). In each yml config under `configs` directory\n(`imagenet_64_cc.yml`, `lsun_bedroom_256.yml` and `lsun_cat_256.yml`) you may change:\n```yaml\nsampling:\n  T_sampling: \u003cdesired_sampling_steps\u003e\n```\n\n## Evaluation\n\nThe [evaluate_CM4IR.sh](evaluate_CM4IR.sh) script can be used for reproducing paper results.\n\n\n## Qualitative Results\n\n![Qualitative Results](figs/qualitative_results.png)\n\nAdditional results can be found in the paper, including PSNR and LPIPS results compared to competitors.\n\n## Citations\nIf you used this repository in your research, please cite the paper:\n```\n@inproceedings{garber2024zero,\n  title={Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond)},\n  author={Garber, Tomer and Tirer, Tom},\n  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},\n  year={2025}\n}\n```\n\nThis implementation is inspired by https://github.com/tirer-lab/DDPG, https://github.com/openai/consistency_models and https://github.com/bahjat-kawar/ddrm.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftirer-lab%2FCM4IR","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftirer-lab%2FCM4IR","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftirer-lab%2FCM4IR/lists"}