{"id":18770121,"url":"https://github.com/doubiiu/aidn","last_synced_at":"2025-04-13T07:31:58.364Z","repository":{"id":149377443,"uuid":"512734294","full_name":"Doubiiu/AIDN","owner":"Doubiiu","description":"[IEEE TIP 2023] Scale-Arbitrary Invertible Image Downscaling (AIDN)","archived":false,"fork":false,"pushed_at":"2023-10-08T14:11:39.000Z","size":18673,"stargazers_count":23,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2023-10-08T15:57:35.430Z","etag":null,"topics":["dynamic-convolution","image","image-rescaling","invertible","super-resolution"],"latest_commit_sha":null,"homepage":"https://doubiiu.github.io/projects/aidn/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Doubiiu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-07-11T11:46:54.000Z","updated_at":"2023-10-08T15:57:36.390Z","dependencies_parsed_at":null,"dependency_job_id":"a799818e-56ea-4133-aa23-4c8fa42eead3","html_url":"https://github.com/Doubiiu/AIDN","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Doubiiu%2FAIDN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Doubiiu%2FAIDN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Doubiiu%2FAIDN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Doubiiu%2FAIDN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Doubiiu","download_url":"https://codeload.github.com/Doubiiu/AIDN/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223573939,"owners_count":17167404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dynamic-convolution","image","image-rescaling","invertible","super-resolution"],"created_at":"2024-11-07T19:18:14.236Z","updated_at":"2024-11-07T19:18:15.076Z","avatar_url":"https://github.com/Doubiiu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## **Scale-Arbitrary Invertible Image Downscaling (AIDN)**\r\n\r\nOfficial PyTorch implementation for the paper:\r\n\r\n\u003e **Scale-Arbitrary Invertible Image Downscaling**\r\n\u003e\r\n\u003e ***IEEE Transactions on Image Processing (TIP) 2023***\r\n\u003e\r\n\u003e Jinbo Xing*, Wenbo Hu*, Menghan Xia, Tien-Tsin Wong    (*joint first authors)\r\n\u003e\r\n\u003e \u003ca href='https://ieeexplore.ieee.org/document/10192538?denied='\u003e\u003cimg src='https://img.shields.io/badge/Paper-IEEExplore-red'\u003e\u003c/a\u003e \u003ca href='https://doubiiu.github.io/projects/aidn/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Video-Green'\u003e\u003c/a\u003e \u003ca href='https://doubiiu.github.io/projects/aidn/compare.html'\u003e\u003cimg src='https://img.shields.io/badge/Online-Demo-Blue'\u003e\u003c/a\u003e\r\n\r\n\u003cp align=\"center\"\u003e\r\n\u003cimg src=\"assets/teaser_README.png\" width=\"98%\"/\u003e\r\n\u003c/p\u003e\r\n\r\n\u003e We present a scale-arbitrary invertible image downscaling network (AIDN) to natively downscale HR images with arbitrary scale factors. Meanwhile, the HR images could be restored with AIDN whenever necessary.\r\n\r\n\u003cp align=\"center\"\u003e\r\n\u003cimg src=\"assets/usecase.png\" width=\"80%\"/\u003e\r\n\u003c/p\u003e\r\n\r\n\u003e Usecase of our AIDN. (a) shows the conventional pipeline of distributing HR images over\r\nsocial media platforms. (b) shows the distribution pipeline with our proposed\r\nAIDN. \u003ci\u003eH\u003c/i\u003e and \u003ci\u003eW\u003c/i\u003e denote the height and width of images; s\u003csub\u003e1\u003c/sub\u003e, ..., s\u003csub\u003en\u003c/sub\u003e are scale factors; and \u003ci\u003eN\u003c/i\u003e stands for the upper-limit resolution of various social media platforms. AIDN allows users to bypass the resolution upper-limit of social media platforms by preventing from auto-downscaling, thus receivers can obtain HR images with more details.\r\n\r\n\u003c!-- ## **TODO**\r\n- [ ] Release an interactive inspection demo. --\u003e\r\n\r\n## **Changelog**\r\n- 2023.09.07 Fix a bug which potentially caused inconsistent quantitative results between the released code and the paper\r\n- 2023.07.30 Release an interactive inspection demo.\r\n- 2023.07.17 Release code and model weights!\r\n\r\n## **Environment**\r\n```\r\nconda create -n AIDN python=3.6.2\r\nconda activate AIDN\r\nconda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch\r\npip install -r requirements.txt\r\n```\r\n\r\n## **Dataset Preparation**\r\nThe training and testing datasets can be downloaded [here](https://github.com/xinntao/BasicSR/blob/master/docs/DatasetPreparation.md). \r\n\r\nFor training, download \u0026 unzip DIV2K dataset, and put `DIV2K_train_HR/` and `DIV2K_valid_HR/` into `Data/`. Fill the path in `dataset/prepare_div2k.py` and execute this script to split the images into patches.\r\n\r\nThe processed datasets should be as below:\r\n```\r\nData/\r\n└── DIV2K/\r\n  ├── DIV2K_valid_HR\r\n  ├── DIV2K_train_HR_patch/\r\n  ├── DIV2K_valid_HR_patch/\r\n    ├── 0801_001.png\r\n    ├── ...\r\n    └── 0900_021.png\r\n  └── list/\r\n    ├── train.txt\r\n    ├── val.txt\r\n    └── test.txt\r\n```\r\n\r\nWe crop the images in testing datasets to make sure their height and width are divisible by 12. The datasets should be as below: \r\n```\r\nData/\r\n├── Set5/\r\n  └──GTmod12/\r\n    ├── xxx.png\r\n    ├── ...\r\n    └── xxx.png\r\n├── Set14/\r\n├── urban100/\r\n├── BSDS100/\r\n├── DIV2K/\r\n└── list/\r\n    ├── DIV2K_val.txt\r\n    ├── ...\r\n    ├── BSDS100_val.txt\r\n    └── DIV2K_val.txt\r\n```\r\nwhere `*.txt` are data lists, whose rows will be `\u003cdataset_name\u003e/GTmod12/\u003cimg_filename\u003e`, e.g. in `BSDS100_val.txt`:\r\n```\r\nBSDS100/GTmod12/101085.png\r\n...\r\nBSDS100/GTmod12/97033.png\r\n```\r\n\r\n## **Train**\r\n```\r\nsh scripts/train.sh \u003cexp_name\u003e \u003cconfig_path\u003e\r\ne.g.: sh scripts/train.sh AIDN_exp01 config/DIV2K/AIDN.yaml\r\n```\r\nNote that, we firstly train the model with a fixed $\\times$ 4 scale for 300 epochs and then on all scale factors for 280 epochs in the paper. We find it can achieve slightly better performance, while this training strategy is just optional.\r\n\r\nAfter training, the log and model weights will be saved in `LOG/DIV2K/\u003cexp_name\u003e`.\r\n\r\n\r\n\r\n## **Benchmark**\r\n```\r\nsh scripts/benchmark.sh \u003cexp_name\u003e \u003cconfig_path\u003e\r\ne.g.: sh scripts/benchmark.sh AIDN_exp01 config/DIV2K/AIDN.yaml\r\n```\r\n\r\n## **Benchmark the pre-trained model weights**\r\nPlease download the pre-trained weights of [AIDN](https://drive.google.com/file/d/1tYlOTQvnsszfLs6cn8p4GQ3Nkzgb4r92/view?usp=drive_link)  and place it in the `LOG/DIV2K/pre-train/` folder, and then run the benchmarking script:\r\n```\r\nsh scripts/AIDN_benchmark.sh config/DIV2K/AIDN_benchmark.yaml\r\n```\r\nNote that here we only provide a small-scale `Set5` dataset for reproduction purpose (actually we don't host the right of redistributing these datasets), you can modify `config/DIV2K/AIDN_benchmark.yaml` to benchmark on more downloaded datasets.\r\n\r\nThe stdout of running this script should be:\r\n```\r\n=\u003e Dataset 'Set5' (x1.5)\r\n==\u003eres_lr:\r\nPSNR: 42.42\r\nSSIM: 0.9870\r\nPSNR-Y: 48.56\r\nSSIM-Y: 0.9962\r\n==\u003eres_sr:\r\nPSNR: 45.26\r\nSSIM: 0.9854\r\nPSNR-Y: 50.61\r\nSSIM-Y: 0.9961\r\n\r\nDataset 'Set5' (x2.5)\r\n==\u003eres_lr:\r\nPSNR: 39.42\r\nSSIM: 0.9851\r\nPSNR-Y: 46.04\r\nSSIM-Y: 0.9960\r\n==\u003eres_sr:\r\nPSNR: 37.43\r\nSSIM: 0.9550\r\nPSNR-Y: 40.77\r\nSSIM-Y: 0.9750\r\n\r\nDataset 'Set5' (x3.5)\r\n==\u003eres_lr:\r\nPSNR: 37.89\r\nSSIM: 0.9853\r\nPSNR-Y: 44.21\r\nSSIM-Y: 0.9960\r\n==\u003eres_sr:\r\nPSNR: 34.35\r\nSSIM: 0.9267\r\nPSNR-Y: 37.25\r\nSSIM-Y: 0.9538\r\n```\r\n\r\nThe JPEG-robust version of AIDN (i.e., [AIDN+](https://drive.google.com/file/d/1l0vsFlbiy3KkOM-dExOHmTQTWpJbo1wO/view?usp=drive_link)) and the weights of [AIDN](https://drive.google.com/file/d/1vicEmixQGrIH815MXN2WPKjKrk6qjUVC/view?usp=sharing) pre-trained with fixed $\\times$ 4 scale are also available.\r\n\r\n## **Citation**\r\nIf you find the code useful for your work, please star this repo and consider citing:\r\n```\r\n@article{xing2023scale,\r\n  title={Scale-arbitrary invertible image downscaling},\r\n  author={Xing, Jinbo and Hu, Wenbo and Xia, Menghan and Wong, Tien-Tsin},\r\n  journal={IEEE Transactions on Image Processing},\r\n  year={2023},\r\n  publisher={IEEE}\r\n}\r\n```\r\n\r\n## **Acknowledgement**\r\nThe code is partially borrowed from [EDSR](https://github.com/sanghyun-son/EDSR-PyTorch), [ArbSR](https://github.com/The-Learning-And-Vision-Atelier-LAVA/ArbSR) and [DiffJPEG](https://github.com/mlomnitz/DiffJPEG). We thank the authors for sharing their code.\r\n\r\n## **Related Work**\r\n- [Invertible Image Rescaling (ECCV 2020 \u0026 IJCV 2022)](https://github.com/pkuxmq/Invertible-Image-Rescaling)\r\n- [Real-time 6K Image Rescaling with Rate-distortion Optimization (CVPR 2023)](https://github.com/AbnerVictor/HyperThumbnail)\r\n- [Towards Bidirectional Arbitrary Image Rescaling: Joint Optimization and Cycle Idempotence (CVPR 2022)](https://arxiv.org/abs/2203.00911)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdoubiiu%2Faidn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdoubiiu%2Faidn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdoubiiu%2Faidn/lists"}