{"id":18613991,"url":"https://github.com/jingyunliang/manet","last_synced_at":"2025-04-10T23:32:13.614Z","repository":{"id":38535746,"uuid":"392310009","full_name":"JingyunLiang/MANet","owner":"JingyunLiang","description":"Official PyTorch code for Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021)","archived":false,"fork":false,"pushed_at":"2021-09-19T07:51:42.000Z","size":1048,"stargazers_count":172,"open_issues_count":15,"forks_count":24,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-25T06:51:12.452Z","etag":null,"topics":["blind-image-sr","blind-image-super-resolution","iccv2021","image-sr","image-super-resolution","kernel-estimation","spatially-variant","super-resolution"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2108.05302","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JingyunLiang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-03T12:28:06.000Z","updated_at":"2025-03-11T13:12:45.000Z","dependencies_parsed_at":"2022-08-18T11:33:29.944Z","dependency_job_id":null,"html_url":"https://github.com/JingyunLiang/MANet","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingyunLiang%2FMANet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingyunLiang%2FMANet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingyunLiang%2FMANet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingyunLiang%2FMANet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JingyunLiang","download_url":"https://codeload.github.com/JingyunLiang/MANet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248316506,"owners_count":21083456,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blind-image-sr","blind-image-super-resolution","iccv2021","image-sr","image-super-resolution","kernel-estimation","spatially-variant","super-resolution"],"created_at":"2024-11-07T03:24:37.280Z","updated_at":"2025-04-10T23:32:11.803Z","avatar_url":"https://github.com/JingyunLiang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution (MANet, ICCV2021)\n\n[![arXiv](https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2108.05302)\n[![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/MANet?style=social)](https://github.com/JingyunLiang/MANet)\n[![download](https://img.shields.io/github/downloads/JingyunLiang/MANet/total.svg)](https://github.com/JingyunLiang/MANet/releases)\n[ \u003ca href=\"https://colab.research.google.com/gist/JingyunLiang/4ed2524d6e08343710ee408a4d997e1c/manet-demo-on-spatially-variant-kernel-estimation.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e](https://colab.research.google.com/gist/JingyunLiang/4ed2524d6e08343710ee408a4d997e1c/manet-demo-on-spatially-variant-kernel-estimation.ipynb)\n\n\n\nThis repository is the official PyTorch implementation of Mutual Affine Network for ***Spatially Variant*** Kernel Estimation in Blind Image Super-Resolution\n([arxiv](https://arxiv.org/pdf/2108.05302.pdf), [supplementary](https://github.com/JingyunLiang/MANet/releases)).\n\n :rocket:  :rocket:  :rocket: **News**:\n  - Aug. 7, 2021: We add an [online Colab demo for MANet kernel estimation \u003ca href=\"https://colab.research.google.com/gist/JingyunLiang/4ed2524d6e08343710ee408a4d997e1c/manet-demo-on-spatially-variant-kernel-estimation.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e](https://colab.research.google.com/gist/JingyunLiang/4ed2524d6e08343710ee408a4d997e1c/manet-demo-on-spatially-variant-kernel-estimation.ipynb)\n  - Sep.06, 2021: See our recent work [SwinIR: Transformer-based image restoration](https://github.com/JingyunLiang/SwinIR).[![arXiv](https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2108.10257)[![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/SwinIR?style=social)](https://github.com/JingyunLiang/SwinIR)[![download](https://img.shields.io/github/downloads/JingyunLiang/SwinIR/total.svg)](https://github.com/JingyunLiang/SwinIR/releases)[ \u003ca href=\"https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e](https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb)\n  - Aug. 17, 2021: See our previous work on [blind SR: Flow-based Kernel Prior with Application to Blind Super-Resolution (FKP), CVPR2021](https://github.com/JingyunLiang/FKP)  [![arXiv](https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2103.15977)\n[![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/FKP?style=social)](https://github.com/JingyunLiang/FKP)*\n - Aug. 17, 2021: See our recent work for [generative modelling of image SR: Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow), ICCV2021](https://github.com/JingyunLiang/HCFlow) [![arXiv](https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2108.05301)\n[![GitHub Stars](https://img.shields.io/github/stars/JingyunLiang/HCFlow?style=social)](https://github.com/JingyunLiang/HCFlow)[![download](https://img.shields.io/github/downloads/JingyunLiang/HCFlow/total.svg)](https://github.com/JingyunLiang/HCFlow/releases)[ \u003ca href=\"https://colab.research.google.com/gist/JingyunLiang/cdb3fef89ebd174eaa43794accb6f59d/hcflow-demo-on-x8-face-image-sr.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e](https://colab.research.google.com/gist/JingyunLiang/cdb3fef89ebd174eaa43794accb6f59d/hcflow-demo-on-x8-face-image-sr.ipynb)\n - Aug. 17, 2021: See our recent work for [real-world image SR: Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (BSRGAN), ICCV2021](https://github.com/cszn/BSRGAN)  [![arXiv](https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/abs/2103.14006)\n[![GitHub Stars](https://img.shields.io/github/stars/cszn/BSRGAN?style=social)](https://github.com/cszn/BSRGAN)\n \n  ---\n\n\u003e Existing blind image super-resolution (SR) methods mostly assume blur kernels are spatially invariant across the whole image. However, such an assumption is rarely applicable for real images whose blur kernels are usually spatially variant due to factors such as object motion and out-of-focus. Hence, existing blind SR methods would inevitably give rise to poor performance in real applications. To address this issue, this paper proposes a mutual affine network (MANet) for spatially variant kernel estimation. Specifically, MANet has two distinctive features. First, it has a moderate receptive field so as to keep the locality of degradation. Second, it involves a new mutual affine convolution (MAConv) layer that enhances feature expressiveness without increasing receptive field, model size and computation burden. This is made possible through exploiting channel interdependence, which applies each channel split with an affine transformation module whose input are the rest channel splits. Extensive experiments on synthetic and real images show that the proposed MANet not only performs favorably for both spatially variant and invariant kernel estimation, but also leads to state-of-the-art blind SR performance when combined with non-blind SR methods.\n\u003e\u003cp align=\"left\"\u003e\n   \u003e \u003cimg width=\"360\" src=\"./illustrations/MANet.png\"\u003e\u003cimg width=\"240\" src=\"./illustrations/MAConv.png\"\u003e\n\u003c/p\u003e\n\n\n\n## Requirements\n- Python 3.7, PyTorch \u003e= 1.6.0, scipy \u003e= 1.6.3 \n- Requirements: opencv-python\n- Platforms: Ubuntu 16.04, cuda-10.0 \u0026 cuDNN v-7.5\n\nNote: this repository is based on [BasicSR](https://github.com/xinntao/BasicSR#memo-codebase-designs-and-conventions). Please refer to their repository for a better understanding of the code framework.\n\n\n## Quick Run\nDownload `stage3_MANet+RRDB_x4.pth` from [release](https://github.com/JingyunLiang/MANet/releases) and put it in `./pretrained_models`. Then, run following command. Or you can go to our [online Colab demo for MANet kernel estimation \u003ca href=\"https://colab.research.google.com/gist/JingyunLiang/4ed2524d6e08343710ee408a4d997e1c/manet-demo-on-spatially-variant-kernel-estimation.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"google colab logo\"\u003e\u003c/a\u003e](https://colab.research.google.com/gist/JingyunLiang/4ed2524d6e08343710ee408a4d997e1c/manet-demo-on-spatially-variant-kernel-estimation.ipynb) to have a try.\n```bash\ncd codes\npython test.py --opt options/test/test_stage3.yml\n```\n---\n\n## Data Preparation\nTo prepare data, put training and testing sets in `./datasets` as `./datasets/DIV2K/HR/0801.png`. Commonly used datasets can be downloaded [here](https://github.com/xinntao/BasicSR/blob/master/docs/DatasetPreparation.md#common-image-sr-datasets).\n\n\n## Training\n\nStep1: to train MANet, run this command:\n\n```bash\npython train.py --opt options/train/train_stage1.yml\n```\n\nStep2: to train non-blind RRDB, run this command:\n\n```bash\npython train.py --opt options/train/train_stage2.yml\n```\n\nStep3: to fine-tune RRDB with MANet, run this command:\n\n```bash\npython train.py --opt options/train/train_stage3.yml\n```\n\nAll trained models can be downloaded from [release](https://github.com/JingyunLiang/MANet/releases). For testing, downloading stage3 models is enough.\n\n\n## Testing\n\nTo test MANet (stage1, kernel estimation only), run this command:\n\n```bash\npython test.py --opt options/test/test_stage1.yml\n```\nTo test RRDB-SFT (stage2, non-blind SR with ground-truth kernel), run this command:\n\n```bash\npython test.py --opt options/test/test_stage2.yml\n```\nTo test MANet+RRDB (stage3, blind SR), run this command:\n\n```bash\npython test.py --opt options/test/test_stage3.yml\n```\nNote: above commands generate LR images on-the-fly. To generate testing sets used in the paper, run this command:\n```bash\npython prepare_testset.py --opt options/test/prepare_testset.yml\n```\n\n## Interactive Exploration of Kernels\nTo explore spaitally variant kernels on an image, use `--save_kernel` and run this command to save kernel:\n\n```bash\npython test.py --opt options/test/test_stage1.yml --save_kernel\n```\nThen, run this command to creat an interactive window:\n```bash\npython interactive_explore.py --path ../results/001_MANet_aniso_x4_test_stage1/toy_dataset1/npz/toy1.npz\n```\n\n## Results\nWe conducted experiments on both spatially variant and invariant blind SR. Please refer to the [paper](https://arxiv.org/abs/2108.05302) and [supp](https://github.com/JingyunLiang/MANet/releases) for results. \n\n## Citation\n    @inproceedings{liang2021mutual,\n      title={Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution},\n      author={Liang, Jingyun and Sun, Guolei and Zhang, Kai and Van Gool, Luc and Timofte, Radu},\n      booktitle={IEEE International Conference on Computer Vision},\n      year={2021}\n    }\n\n## License \u0026 Acknowledgement\n\nThis project is released under the Apache 2.0 license. The codes are based on [BasicSR](https://github.com/xinntao/BasicSR), [MMSR](https://github.com/open-mmlab/mmediting), [IKC](https://github.com/yuanjunchai/IKC) and [KAIR](https://github.com/cszn/KAIR). Please also follow their licenses. Thanks for their great works.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjingyunliang%2Fmanet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjingyunliang%2Fmanet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjingyunliang%2Fmanet/lists"}