{"id":22069079,"url":"https://github.com/HPLQAQ/SAM-Deblur","last_synced_at":"2025-07-24T07:32:03.472Z","repository":{"id":191856318,"uuid":"685544578","full_name":"HPLQAQ/SAM-Deblur","owner":"HPLQAQ","description":"Implementation of the paper SAM-Deblur: Let Segment Anything Boost Image Deblurring(ICASSP2024)","archived":false,"fork":false,"pushed_at":"2023-12-22T07:06:25.000Z","size":869,"stargazers_count":11,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2023-12-22T08:23:52.636Z","etag":null,"topics":["image-deblurring","image-restoration","ood-generalization","plug-and-play","segment-anything"],"latest_commit_sha":null,"homepage":"https://hplqaq.github.io/projects/sam-deblur","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HPLQAQ.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-08-31T13:21:36.000Z","updated_at":"2023-12-19T13:25:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"d7f456bc-54e4-48b9-b83a-b5d86c64db46","html_url":"https://github.com/HPLQAQ/SAM-Deblur","commit_stats":null,"previous_names":["hplqaq/sam-deblur"],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HPLQAQ%2FSAM-Deblur","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HPLQAQ%2FSAM-Deblur/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HPLQAQ%2FSAM-Deblur/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HPLQAQ%2FSAM-Deblur/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HPLQAQ","download_url":"https://codeload.github.com/HPLQAQ/SAM-Deblur/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227421420,"owners_count":17775011,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["image-deblurring","image-restoration","ood-generalization","plug-and-play","segment-anything"],"created_at":"2024-11-30T20:04:39.382Z","updated_at":"2024-11-30T20:07:37.469Z","avatar_url":"https://github.com/HPLQAQ.png","language":"Python","funding_links":[],"categories":["Paper List"],"sub_categories":["Follow-up Papers"],"readme":"## SAM-Deblur: Let Segment Anything Boost Image Deblurring\n---\n\nImplementation of the paper [SAM-Deblur: Let Segment Anything Boost Image Deblurring(ICASSP2024)](https://hplqaq.github.io/projects/sam-deblur)\n\n##### [Siwei Li](https://hplqaq.github.io/)\\*, [Mingxuan Liu](https://arktis2022.github.io/)\\*, Yating Zhang, Shu Chen, [Haoxiang Li](https://lihaoxiang-20.github.io/), Zifei Dou, [Hong Chen](http://hongchen.ime.tsinghua.edu.cn/)\n\n[[`Project`](https://hplqaq.github.io/projects/sam-deblur)] [[`Paper`](https://arxiv.org/abs/2309.02270v2)] [[`BibTeX`](https://hplqaq.github.io/projects/sam-deblur.assets/bib_sam_deblur.txt)]\n\n![SAM design](assets/SAM-Deblur-pipeline.png)\n\nTodo:\n- [x] Full code release with instruction\n- [x] Training and testing options\n- [x] Pretrained models and prepared data\n- [x] Instructions for data preparation code\n\n### Installation\nThis implementation is based on [BasicSR](https://github.com/xinntao/BasicSR) which is an open source toolbox for image/video restoration tasks.\n\n```python\npython 3.10.13\npytorch 1.13.1\ncuda 11.7\n```\n\n```\nconda create -n sam-deblur python=3.10\nconda activate sam-deblur\nconda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia\n```\n\n```\ngit clone https://github.com/HPLQAQ/SAM-Deblur.git\ncd SAM-Deblur\npip install -r requirements.txt\npip install -e .\n```\n### Prepare Data and Pre-trained Models\n\nDownload pre-trained models for experiments from [Baidu Netdisk](https://pan.baidu.com/s/109TCMu7_ZuedUz8qG-Q0IQ?pwd=hplv)|[Onedrive](https://1drv.ms/f/s!Ar-cR-XYHDTtgt8yuujyjIENc_B11g?e=LzoHrG).  \nPut under **experiments/pretrained_models**.\n\nDownload prepared data for experiments from [Baidu Netdisk](https://pan.baidu.com/s/1jHgwKqFuyyy8yixByiEfCg?pwd=hplv)|[Onedrive](https://1drv.ms/f/s!Ar-cR-XYHDTtgt8pYy2MBqQvroXl-Q?e=U7o4xc).  \nIf you only want to run our deblur pipeline, download test/val datasets(GoPro, RealBlurJ, REDS, ReLoBlur provided). if you want to train the model yourself, download train datasets(GoPro provided). Check [datasets README](datasets/README.md) for standard dataset structure.\nUnzip data and put under **datasets** dir for experiments.\n\nPrepared data comes from original datasets which are processed using code under **scripts/data_preparation**. *Use datasets uploaded above and your don't have to run the scripts your self.*\n- **scripts/data_preparation/1_create_masks_with_sam.py**: Create masks for concat method.\n- **scripts/data_preparation/2_masks_to_grouped_masks.py**: Create grouped_masks from masks for our MAP method.\n- **scripts/crop_\\*\\*\\***: Create crops from inputs for training.\n- **scripts/lmdb_\\*\\*\\***: Create lmdb from original disk for faster data loading. (not necessary)\n\n### Quick Start\n\nTest the pretrained model on REDS(can be any of the four datasets) with follow command. Visualization will be under directory **results**.\n\n```\npython basicsr/test.py -opt options/test/REDS/SegNAFNet.yml\n```\n\nOptions' corresponding method see [pretrained_models README](experiments/pretrained_models/README.md)\n\nDetailed instructions under [scripts/test.md](scripts/test.md).  \nTo train model, follow instructions under [scripts/train.md](scripts/train.md).\n\nThe core implementation is at [SegNAFNet_arch.py](basicsr/models/archs/SegNAFNet_arch.py)\n\n### Results\n\nBest results are highlighted in bold. **w/o SAM**: Not using SAM priors, **CAT**: concatenation method, **MAP**: Using SAM-Deblur framework w/o mask dropout, **Ours**: Using SAM-Deblur framework.  \n**Mode Collapse Rate** (MCR) is calculated using a threshold-based method. Specifically, when \\\\( PSNR(I_bm, I_gt) - PSNR(I_dm, I_gt) \u003e 3 \\\\) (where \\\\( I_gt \\\\) is the ground truth), we consider the model to have undergone \"mode collapse\". A lower MCR suggests stronger generalization capabilities of the model.\n\n| Methods | GoPro (PSNR↑ / SSIM↑) | RealBlurJ (PSNR↑ / SSIM↑ / MCR↓) | REDS (PSNR↑ / SSIM↑ / MCR↓) | ReLoBlur (PSNR↑ / SSIM↑ / MCR↓) |\n|---------|-----------------------|----------------------------------|-----------------------------|---------------------------------|\n| w/o SAM | 32.85 / 0.960         | 26.57 / 0.866 / 0.20%           | 25.97 / 0.844 / 3.80%       | 25.26 / 0.687 / 54.68%         |\n| CAT     | **32.88 / 0.961**     | 26.55 / 0.863 / 0.31%           | 26.65 / 0.865 / 2.57%       | 29.77 / 0.882 / 58.73%         |\n| MAP     | 32.82 / 0.960         | 26.57 / 0.866 / 0.31%           | 26.81 / 0.865 / 0.40%       | 30.86 / 0.897 / 55.44%         |\n| **Ours**| 32.83 / 0.960         | **26.62 / 0.867 / 0.00%**       | **26.93 / 0.868 / 0.20%**   | **32.29 / 0.903 / 13.92%**     |\n\n### Citations\nIf SAM-Deblur helps your research or work, please consider citing SAM-Deblur.\n\n```\n@inproceedings{Li2023SAMDeblur,\n  author       = {Siwei Li and\n                  Mingxuan Liu and \n                  Yating Zhang and \n                  Shu Chen and \n                  Haoxiang Li and \n                  Zifei Dou and \n                  Hong Chen},\n  title        = {SAM-Deblur: Let Segment Anything Boost Image Deblurring},\n  booktitle    = {ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},\n  year         = {2024}\n  organization = {IEEE}\n}\n```\n\n### Contact\nIf you have any questions, feel free to contact hplv@foxmail.com.\n\n### License and Acknowledgement\n\nThis project is under the Apache 2.0 license based on [BasicSR](https://github.com/xinntao/BasicSR) which is under the Apache 2.0 license. Thanks to the codes from [NAFNet](https://github.com/megvii-research/NAFNet) and awesome Segment Anything Model from [segment-anything](https://github.com/facebookresearch/segment-anything).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHPLQAQ%2FSAM-Deblur","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHPLQAQ%2FSAM-Deblur","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHPLQAQ%2FSAM-Deblur/lists"}