{"id":26507637,"url":"https://github.com/guaishou74851/idm","last_synced_at":"2025-04-09T16:03:26.839Z","repository":{"id":239289953,"uuid":"799119934","full_name":"Guaishou74851/IDM","owner":"Guaishou74851","description":"(TPAMI 2025) Invertible Diffusion Models for Compressed Sensing [PyTorch]","archived":false,"fork":false,"pushed_at":"2025-03-09T06:48:09.000Z","size":3007,"stargazers_count":124,"open_issues_count":6,"forks_count":12,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-09T16:02:44.951Z","etag":null,"topics":["algorithm-unrolling","compressed-sensing","compressive-sampling","compressive-sensing","computational-imaging","computer-vision","deep-learning","deep-neural-networks","deep-unfolding","deep-unfolding-model","deep-unrolling","diffusion-models","image-reconstruction","image-restoration","invertible-neural-networks","python","pytorch","single-pixel-imaging"],"latest_commit_sha":null,"homepage":"https://ieeexplore.ieee.org/document/10874182","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Guaishou74851.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-11T08:33:23.000Z","updated_at":"2025-04-09T09:49:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"a40583e4-529a-408f-a36f-d4e90ec7df43","html_url":"https://github.com/Guaishou74851/IDM","commit_stats":null,"previous_names":["guaishou74851/idm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Guaishou74851%2FIDM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Guaishou74851%2FIDM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Guaishou74851%2FIDM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Guaishou74851%2FIDM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Guaishou74851","download_url":"https://codeload.github.com/Guaishou74851/IDM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248065290,"owners_count":21041871,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithm-unrolling","compressed-sensing","compressive-sampling","compressive-sensing","computational-imaging","computer-vision","deep-learning","deep-neural-networks","deep-unfolding","deep-unfolding-model","deep-unrolling","diffusion-models","image-reconstruction","image-restoration","invertible-neural-networks","python","pytorch","single-pixel-imaging"],"created_at":"2025-03-20T23:29:26.805Z","updated_at":"2025-04-09T16:03:26.820Z","avatar_url":"https://github.com/Guaishou74851.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# (TPAMI 2025) Invertible Diffusion Models for Compressed Sensing [PyTorch]\n\n[![IEEE-Xplore](https://img.shields.io/badge/IEEE_Xplore-Paper-\u003cCOLOR\u003e.svg)](https://ieeexplore.ieee.org/document/10874182) [![icon](https://img.shields.io/badge/ArXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2403.17006) ![visitors](https://visitor-badge.laobi.icu/badge?page_id=Guaishou74851.IDM)\n\n[Bin Chen](https://scholar.google.com/citations?user=aZDNm98AAAAJ), [Zhenyu Zhang](https://scholar.google.com/citations?user=4TbicrcAAAAJ), [Weiqi Li](https://scholar.google.com/citations?user=SIkQdEsAAAAJ), [Chen Zhao](https://chenzhao.netlify.app/)†, [Jiwen Yu](https://vvictoryuki.github.io/website/), Shijie Zhao, [Jie Chen](https://aimia-pku.github.io/), and [Jian Zhang](https://jianzhang.tech/)\n\n*School of Electronic and Computer Engineering, Peking University, Shenzhen, China.*\n\n*King Abdullah University of Science and Technology, Thuwal, Saudi Arabia.*\n\n*ByteDance Inc, Shenzhen, China.*\n\n† Corresponding author\n\nAccepted for publication in [IEEE Transactions on Pattern Analysis and Machine Intelligence](https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=34) (TPAMI) 2025.\n\n⭐ If IDM is helpful to you, please star this repo. Thanks! 🤗\n\n## 📝 Abstract\n\nWhile deep neural networks (NN) significantly advance image compressed sensing (CS) by improving reconstruction quality, the necessity of training current CS NNs from scratch constrains their effectiveness and hampers rapid deployment. Although recent methods utilize pre-trained diffusion models for image reconstruction, they struggle with slow inference and restricted adaptability to CS. To tackle these challenges, this paper proposes **I**nvertible **D**iffusion **M**odels (**IDM**), a novel efficient, end-to-end diffusion-based CS method. IDM repurposes a large-scale diffusion sampling process as a reconstruction model, and fine-tunes it end-to-end to recover original images directly from CS measurements, moving beyond the traditional paradigm of one-step noise estimation learning. To enable such memory-intensive end-to-end fine-tuning, we propose a novel two-level invertible design to transform both (1) multi-step sampling process and (2) noise estimation U-Net in each step into invertible networks. As a result, most intermediate features are cleared during training to reduce up to 93.8% GPU memory. In addition, we develop a set of lightweight modules to inject measurements into noise estimator to further facilitate reconstruction. Experiments demonstrate that IDM outperforms existing state-of-the-art CS networks by up to 2.64dB in PSNR. Compared to the recent diffusion-based approach DDNM, our IDM achieves up to 10.09dB PSNR gain and 14.54 times faster inference. Code is available at https://github.com/Guaishou74851/IDM.\n\n## 🍭 Overview\n\n![teaser](figs/teaser.png)\n\n![framework](figs/framework.png)\n\n## ⚙ Environment\n\n```shell\ntorch==2.3.1+cu121\ndiffusers==0.30.2\ntransformers==4.44.2\nnumpy==1.26.3\nopencv-python==4.10.0\nscikit-image==0.24.0\n```\n\n## ⚡ Test\n\nDownload the pretrained models ([Google Drive](https://drive.google.com/file/d/1UzUg0lFqwWfmeXi8gqAOeQA3yt4HpPNy/view?usp=sharing), [PKU Disk 北大网盘](https://disk.pku.edu.cn/link/AA0B0294E9BCF64185B677BDF0951A7D54)) and put the `weight` directory into `./`, then run the following command:\n\n```shell\npython test.py --cs_ratio=0.1/0.3/0.5 --testset_name=Set11/CBSD68/Urban100/DIV2K\n```\n\nThe reconstructed images will be in `./result`.\n\nThe test sets CBSD68, Urban100, and DIV2K are available at https://github.com/Guaishou74851/SCNet/tree/main/data.\n\nFor easy comparison, test results of various existing image CS methods are available on [Google Drive](https://drive.google.com/drive/folders/1Lif_7N_bCyILFLac5JcOtJ9cWpGBNVCd) and [PKU Disk](https://disk.pku.edu.cn/link/AA1C2D8A08050744449CBFCAB51A846B2D).\n\n## 🔥 Train\n\nDownload the dataset of [Waterloo Exploration Database](https://kedema.org/project/exploration/index.html) ([Google Drive](https://drive.google.com/file/d/1TOg7BZE1XsJ7l2VzMoqFRAETk7OLcv75/view?usp=drive_link), [PKU Disk 北大网盘](https://disk.pku.edu.cn/link/AAD0DCBBD65D744526921B334ED2AB4F76)) and put the `pristine_images` directory (containing 4744 `.bmp` image files) into `./data`, then run the following command:\n\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.run --nproc_per_node=4 --master_port=23333 train.py --cs_ratio=0.1/0.3/0.5\n```\n\nThe log and model files will be in `./log` and `./weight`, respectively.\n\n## 😍 Results\n\n![res1](figs/res1.png)\n\n![res2](figs/res2.png)\n\n## 🎓 Citation\n\nIf you find the code helpful in your research or work, please cite the following paper:\n\n```latex\n@article{chen2025invertible,\n  title={Invertible Diffusion Models for Compressed Sensing},\n  author={Chen, Bin and Zhang, Zhenyu and Li, Weiqi and Zhao, Chen and Yu, Jiwen and Zhao, Shijie and Chen, Jie and Zhang, Jian},\n  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},\n  year={2025},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguaishou74851%2Fidm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguaishou74851%2Fidm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguaishou74851%2Fidm/lists"}