{"id":17349619,"url":"https://github.com/researchmm/AOT-GAN-for-Inpainting","last_synced_at":"2025-02-26T02:32:00.564Z","repository":{"id":45207901,"uuid":"352255046","full_name":"researchmm/AOT-GAN-for-Inpainting","owner":"researchmm","description":"[TVCG'2023] AOT-GAN for High-Resolution Image Inpainting (codebase for image inpainting)","archived":false,"fork":false,"pushed_at":"2024-05-08T05:03:48.000Z","size":25997,"stargazers_count":429,"open_issues_count":14,"forks_count":69,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-16T18:18:05.907Z","etag":null,"topics":["codebase","high-resolution","image-inpainting","multi-scale"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2104.01431","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/researchmm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-28T06:10:19.000Z","updated_at":"2024-10-14T07:09:13.000Z","dependencies_parsed_at":"2024-05-08T06:35:23.431Z","dependency_job_id":null,"html_url":"https://github.com/researchmm/AOT-GAN-for-Inpainting","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/researchmm%2FAOT-GAN-for-Inpainting","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/researchmm%2FAOT-GAN-for-Inpainting/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/researchmm%2FAOT-GAN-for-Inpainting/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/researchmm%2FAOT-GAN-for-Inpainting/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/researchmm","download_url":"https://codeload.github.com/researchmm/AOT-GAN-for-Inpainting/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240780777,"owners_count":19856425,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["codebase","high-resolution","image-inpainting","multi-scale"],"created_at":"2024-10-15T16:56:23.644Z","updated_at":"2025-02-26T02:32:00.533Z","avatar_url":"https://github.com/researchmm.png","language":"Python","funding_links":[],"categories":["Table of Contents"],"sub_categories":["DETR变种"],"readme":"# AOT-GAN for High-Resolution Image Inpainting\n![aotgan](https://github.com/researchmm/AOT-GAN-for-Inpainting/blob/master/docs/aotgan.PNG?raw=true)\n### [Arxiv Paper](https://arxiv.org/abs/2104.01431) |\n\nAOT-GAN: Aggregated Contextual Transformations for High-Resolution Image Inpainting\u003cbr\u003e\n[Yanhong Zeng](https://sites.google.com/view/1900zyh),  [Jianlong Fu](https://jianlong-fu.github.io/), [Hongyang Chao](https://scholar.google.com/citations?user=qnbpG6gAAAAJ\u0026hl),  and [Baining Guo](https://www.microsoft.com/en-us/research/people/bainguo/).\u003cbr\u003e\n\n\n\u003c!-- ------------------------------------------------ --\u003e\n## Citation\nIf any part of our paper and code is helpful to your work,\nplease generously cite and star us :kissing_heart: :kissing_heart: :kissing_heart: !\n\n```\n@inproceedings{yan2021agg,\n  author = {Zeng, Yanhong and Fu, Jianlong and Chao, Hongyang and Guo, Baining},\n  title = {Aggregated Contextual Transformations for High-Resolution Image Inpainting},\n  booktitle = {Arxiv},\n  pages={-},\n  year = {2020}\n}\n```\n\n\n\u003c!-- ---------------------------------------------------- --\u003e\n## Introduction\nDespite some promising results, it remains challenging for existing image inpainting approaches to fill in large missing regions in high resolution images (e.g., 512x512). We analyze that the difficulties mainly drive from simultaneously inferring missing contents and synthesizing fine-grained textures for a extremely large missing region.\nWe propose a GAN-based model that improves performance by,\n1) **Enhancing context reasoning by AOT Block in the generator.** The AOT blocks aggregate contextual transformations with different receptive fields, allowing to capture both informative distant contexts and rich patterns of interest for context reasoning.\n2) **Enhancing texture synthesis by SoftGAN in the discriminator.**  We improve the training of the discriminator by a tailored mask-prediction task. The enhanced discriminator is optimized to distinguish the detailed appearance of real and synthesized patches, which can in turn facilitate the generator to synthesize more realistic textures.\n\n\n\u003c!-- ------------------------------------------------ --\u003e\n## Results\n![face_object](https://github.com/researchmm/AOT-GAN-for-Inpainting/blob/master/docs/face_object.PNG?raw=true)\n![logo](https://github.com/researchmm/AOT-GAN-for-Inpainting/blob/master/docs/logo.PNG?raw=true)\n\n\n\u003c!-- -------------------------------- --\u003e\n## Prerequisites\n* python 3.8.8\n* [pytorch](https://pytorch.org/) (tested on Release 1.8.1)\n\n\u003c!-- --------------------------------- --\u003e\n## Installation\n\nClone this repo.\n\n```\ngit clone git@github.com:researchmm/AOT-GAN-for-Inpainting.git\ncd AOT-GAN-for-Inpainting/\n```\n\nFor the full set of required Python packages, we suggest create a Conda environment from the provided YAML, e.g.\n\n```\nconda env create -f environment.yml\nconda activate inpainting\n```\n\n\u003c!-- --------------------------------- --\u003e\n## Datasets\n\n1. download images and masks\n2. specify the path to training data by `--dir_image` and `--dir_mask`.\n\n\n\n\u003c!-- -------------------------------------------------------- --\u003e\n## Getting Started\n\n1. Training:\n    * Our codes are built upon distributed training with Pytorch.\n    * Run\n    ```\n    cd src\n    python train.py\n    ```\n2. Resume training:\n    ```\n    cd src\n    python train.py --resume\n    ```\n3. Testing:\n    ```\n    cd src\n    python test.py --pre_train [path to pretrained model]\n    ```\n4. Evaluating:\n    ```\n    cd src\n    python eval.py --real_dir [ground truths] --fake_dir [inpainting results] --metric mae psnr ssim fid\n    ```\n\n\u003c!-- ------------------------------------------------------------------- --\u003e\n## Pretrained models\n[CELEBA-HQ](https://drive.google.com/drive/folders/1Zks5Hyb9WAEpupbTdBqsCafmb25yqsGJ?usp=sharing) |\n[Places2](https://drive.google.com/drive/folders/1bSOH-2nB3feFRyDEmiX81CEiWkghss3i?usp=sharing)\n\nDownload the model dirs and put it under `experiments/`\n\n\n\u003c!-- ------------------------------------------------------------------- --\u003e\n## Demo\n\n1. Download the pre-trained model parameters and put it under `experiments/`\n2. Run by\n```\ncd src\npython demo.py --dir_image [folder to images]  --pre_train [path to pre_trained model] --painter [bbox|freeform]\n```\n3. Press '+' or '-' to control the thickness of painter.\n4. Press 'r' to reset mask; 'k' to keep existing modifications; 's' to save results.\n5. Press space to perform inpainting; 'n' to move to next image; 'Esc' to quit demo.\n\n\n![face](https://github.com/researchmm/AOT-GAN-for-Inpainting/blob/master/docs/face.gif?raw=true)\n![logo](https://github.com/researchmm/AOT-GAN-for-Inpainting/blob/master/docs/logo.gif?raw=true)\n\n\n\n\u003c!-- ------------------------ --\u003e\n## TensorBoard\nVisualization on TensorBoard for training is supported.\n\nRun `tensorboard --logdir [log_folder] --bind_all` and open browser to view training progress.\n\n## LICENSE\nThis project is released under the Apache 2.0 license. Please see the LICENSE file for more information.\n\n\u003c!-- ------------------------ --\u003e\n## Acknowledgements\n\nWe would like to thank [edge-connect](https://github.com/knazeri/edge-connect), [EDSR_PyTorch](https://github.com/sanghyun-son/EDSR-PyTorch).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresearchmm%2FAOT-GAN-for-Inpainting","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fresearchmm%2FAOT-GAN-for-Inpainting","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresearchmm%2FAOT-GAN-for-Inpainting/lists"}