{"id":13737787,"url":"https://github.com/Ha0Tang/AttentionGAN","last_synced_at":"2025-05-08T15:31:43.352Z","repository":{"id":44332951,"uuid":"196018884","full_name":"Ha0Tang/AttentionGAN","owner":"Ha0Tang","description":"AttentionGAN for Unpaired Image-to-Image Translation \u0026 Multi-Domain Image-to-Image Translation","archived":false,"fork":false,"pushed_at":"2023-07-06T21:30:54.000Z","size":216835,"stargazers_count":628,"open_issues_count":17,"forks_count":94,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-08-04T03:11:10.381Z","etag":null,"topics":["adversarial-networks","attention-model","computer-vision","cyclegan","deep-learning","gans","geometry","ijcnn","image-generation","image-to-image-translation","image-translation","multi-domain","pytorch","unpaired"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ha0Tang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-07-09T13:54:20.000Z","updated_at":"2024-07-07T21:04:47.000Z","dependencies_parsed_at":"2024-01-07T05:59:54.034Z","dependency_job_id":"5fce8d45-b88e-40b9-b5c7-4f8c60814f05","html_url":"https://github.com/Ha0Tang/AttentionGAN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ha0Tang%2FAttentionGAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ha0Tang%2FAttentionGAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ha0Tang%2FAttentionGAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ha0Tang%2FAttentionGAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ha0Tang","download_url":"https://codeload.github.com/Ha0Tang/AttentionGAN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224742387,"owners_count":17362229,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-networks","attention-model","computer-vision","cyclegan","deep-learning","gans","geometry","ijcnn","image-generation","image-to-image-translation","image-translation","multi-domain","pytorch","unpaired"],"created_at":"2024-08-03T03:02:00.862Z","updated_at":"2025-05-08T15:31:43.332Z","avatar_url":"https://github.com/Ha0Tang.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[![License CC BY-NC-SA 4.0](https://img.shields.io/badge/license-CC4.0-blue.svg)](https://github.com/Ha0Tang/AttentionGAN/blob/master/LICENSE.md)\n![Python 3.6](https://img.shields.io/badge/python-3.6-green.svg)\n![Packagist](https://img.shields.io/badge/Pytorch-0.4.1-red.svg)\n![Last Commit](https://img.shields.io/github/last-commit/Ha0Tang/AttentionGAN)\n[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-blue.svg)]((https://github.com/Ha0Tang/AttentionGAN/graphs/commit-activity))\n![Contributing](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)\n![Ask Me Anything !](https://img.shields.io/badge/Ask%20me-anything-1abc9c.svg)\n\n# AttentionGAN-v2 for Unpaired Image-to-Image Translation\n\n## AttentionGAN-v2 Framework\nThe proposed generator learns both foreground and background attentions. It uses the foreground attention to select from the generated output for the foreground regions, while uses the background attention to maintain the background information from the input image. Please refer to our papers for more details.\n\n![Framework](./imgs/framework.jpg)\n\n## Comparsion with State-of-the-Art Methods\n### Selfie To Anime Translation\n![Result](./imgs/selfie_results.jpg)\n\n### Horse to Zebra Translation\n![Result](./imgs/h2z_comparsion2.jpg)\n\u003cbr\u003e\n![Result](./imgs/h2z_comparsion3.jpg)\n\n### Zebra to Horse  Translation\n![Result](./imgs/z2h_comparsion.jpg)\n\n### Apple to Orange Translation\n![Result](./imgs/a2o_comparsion.jpg)\n\n### Orange to Apple  Translation\n![Result](./imgs/o2a_comparsion.jpg)\n\n### Map to Aerial Photo Translation\n![Result](./imgs/m2l_comparison.jpg)\n\n### Aerial Photo to Map Translation\n![Result](./imgs/l2m_comparison.jpg)\n\n### Style Transfer\n![Result](./imgs/style_transfer_results.jpg)\n\n## Visualization of Learned Attention Masks  \n### Selfie to Anime Translation\n![Result](./imgs/selfie_attention.jpg)\n\n### Horse to Zebra Translation\n![Attention](./imgs/h2z_attention_maps.jpg)\n\n### Zebra to Horse Translation\n![Attention](./imgs/z2h_attention_maps.jpg)\n\n### Apple to Orange Translation\n![Attention](./imgs/a2o_attention_maps.jpg)\n\n### Orange to Apple  Translation\n![Attention](./imgs/o2a_attention_maps.jpg)\n\n### Map to Aerial Photo Translation\n![Attention](./imgs/l2m_attention_maps.jpg)\n\n### Aerial Photo to Map Translation\n![Attention](./imgs/m2l_attention_maps.jpg)\n\n### [Extended Paper](https://arxiv.org/abs/1911.11897) | [Conference Paper](https://arxiv.org/abs/1903.12296)\n\nAttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks.\u003cbr\u003e\n[Hao Tang](http://disi.unitn.it/~hao.tang/)\u003csup\u003e1\u003c/sup\u003e, [Hong Liu](https://scholar.google.com/citations?user=4CQKG8oAAAAJ\u0026hl=en)\u003csup\u003e2\u003c/sup\u003e, [Dan Xu](http://www.robots.ox.ac.uk/~danxu/)\u003csup\u003e3\u003c/sup\u003e, [Philip H.S. Torr](https://scholar.google.com/citations?user=kPxa2w0AAAAJ\u0026hl=en)\u003csup\u003e3\u003c/sup\u003e and [Nicu Sebe](http://disi.unitn.it/~sebe/)\u003csup\u003e1\u003c/sup\u003e. \u003cbr\u003e \n\u003csup\u003e1\u003c/sup\u003eUniversity of Trento, Italy, \u003csup\u003e2\u003c/sup\u003ePeking University, China, \u003csup\u003e3\u003c/sup\u003eUniversity of Oxford, UK.\u003cbr\u003e\nIn TNNLS 2021 \u0026 IJCNN 2019 Oral. \u003cbr\u003e\nThe repository offers the official implementation of our paper in PyTorch.\n\n#### Are you looking for AttentionGAN-v1 for Unpaired Image-to-Image Translation?\n\u003e [Paper](https://arxiv.org/abs/1903.12296) | [Code](./AttentionGAN-v1)\n\n#### Are you looking for AttentionGAN-v1 for Multi-Domain Image-to-Image Translation?\n\u003e [Paper](https://arxiv.org/abs/1903.12296) | [Code](./AttentionGAN-v1-multi)\n\n##### Facial Expression-to-Expression Translation\n![Result](./imgs/RaFD_results.jpg)\nOrder: The Learned Attention Masks, The Learned Content Masks, Final Results\n\n##### Facial Attribute Transfer\n![Attention](./imgs/celeba_attention.jpg)\nOrder: The Learned Attention Masks, The Learned Content Masks, Final Results\n\n![Result](./imgs/CelebA_results.jpg)\nOrder: The Learned Attention Masks, AttentionGAN, StarGAN\n\n### [License](./LICENSE.md)\n\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e\u003cimg alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003e\nCopyright (C) 2019 University of Trento, Italy.\n\nAll rights reserved.\nLicensed under the [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode) (**Attribution-NonCommercial-ShareAlike 4.0 International**)\n\nThe code is released for academic research use only. For commercial use, please contact [bjdxtanghao@gmail.com](bjdxtanghao@gmail.com).\n\n## Installation\n\nClone this repo.\n```bash\ngit clone https://github.com/Ha0Tang/AttentionGAN\ncd AttentionGAN/\n```\n\nThis code requires PyTorch 0.4.1+ and python 3.6.9+. Please install dependencies by\n```bash\npip install -r requirements.txt (for pip users)\n```\nor \n\n```bash\n./scripts/conda_deps.sh (for Conda users)\n```\n\nTo reproduce the results reported in the paper, you would need an NVIDIA Tesla V100 with 16G memory.\n\n## Dataset Preparation\nDownload the datasets using the following script. Please cite their paper if you use the data. Try twice if it fails the first time!\n```\nsh ./datasets/download_cyclegan_dataset.sh dataset_name\n```\nThe selfie2anime dataset can be download [here](https://drive.google.com/file/d/1xOWj1UVgp6NKMT3HbPhBbtq2A4EDkghF/view).\n\n## AttentionGAN Training/Testing\n- Download a dataset using the previous script (e.g., horse2zebra).\n- To view training results and loss plots, run `python -m visdom.server` and click the URL [http://localhost:8097](http://localhost:8097).\n- Train a model:\n```\nsh ./scripts/train_attentiongan.sh\n```\n- To see more intermediate results, check out `./checkpoints/horse2zebra_attentiongan/web/index.html`.\n- How to continue train? Append `--continue_train --epoch_count xxx` on the command line.\n- Test the model:\n```\nsh ./scripts/test_attentiongan.sh\n```\n- The test results will be saved to a html file here: `./results/horse2zebra_attentiongan/latest_test/index.html`.\n\n## Generating Images Using Pretrained Model\n- You need download a pretrained model (e.g., horse2zebra) with the following script:\n```\nsh ./scripts/download_attentiongan_model.sh horse2zebra\n```\n- The pretrained model is saved at `./checkpoints/{name}_pretrained/latest_net_G.pth`. \n- Then generate the result using\n```\npython test.py --dataroot ./datasets/horse2zebra --name horse2zebra_pretrained --model attention_gan --dataset_mode unaligned --norm instance --phase test --no_dropout --load_size 256 --crop_size 256 --batch_size 1 --gpu_ids 0 --num_test 5000 --epoch latest --saveDisk\n```\nThe results will be saved at `./results/`. Use `--results_dir {directory_path_to_save_result}` to specify the results directory. Note that if you want to save the intermediate results and have enough disk space, remove `--saveDisk` on the command line.\n\n- For your own experiments, you might want to specify --netG, --norm, --no_dropout to match the generator architecture of the trained model.\n\n### Image Translation with Geometric Changes Between Source and Target Domains\nFor instance, if you want to run experiments of Selfie to Anime Translation. Usage: replace `attention_gan_model.py` and `networks` with the ones in the `AttentionGAN-geo` folder.\n\n### Test the Pretrained Model \nDownload data and pretrained model according above instructions.\n\n`python test.py --dataroot ./datasets/selfie2anime/ --name selfie2anime_pretrained --model attention_gan --dataset_mode unaligned --norm instance --phase test --no_dropout --load_size 256 --crop_size 256 --batch_size 1 --gpu_ids 0 --num_test 5000 --epoch latest`\n\n### Train a New Model\n`python train.py --dataroot ./datasets/selfie2anime/ --name selfie2anime_attentiongan --model attention_gan --dataset_mode unaligned --pool_size 50 --no_dropout --norm instance --lambda_A 10 --lambda_B 10 --lambda_identity 0.5 --load_size 286 --crop_size 256 --batch_size 4 --niter 100 --niter_decay 100 --gpu_ids 0 --display_id 0 --display_freq 100 --print_freq 100`\n\n### Test the Trained Model\n`python test.py --dataroot ./datasets/selfie2anime/ --name selfie2anime_attentiongan --model attention_gan --dataset_mode unaligned --norm instance --phase test --no_dropout --load_size 256 --crop_size 256 --batch_size 1 --gpu_ids 0 --num_test 5000 --epoch latest`\n\n## Evaluation Code\n- [FID](https://github.com/bioinf-jku/TTUR): Official Implementation\n- [KID](https://github.com/taki0112/GAN_Metrics-Tensorflow) or [Here](https://github.com/Ha0Tang/AttentionGAN/tree/master/scripts/GAN_Metrics-Tensorflow): Suggested by [UGATIT](https://github.com/taki0112/UGATIT/issues/64). \n  Install Steps: `conda create -n python36 pyhton=3.6 anaconda` and `pip install --ignore-installed --upgrade tensorflow==1.13.1`. If you encounter the issue `AttributeError: module 'scipy.misc' has no attribute 'imread'`, please do `pip install scipy==1.1.0`.\n\n## Citation\nIf you use this code for your research, please cite our papers.\n```\n@article{tang2021attentiongan,\n  title={AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks},\n  author={Tang, Hao and Liu, Hong and Xu, Dan and Torr, Philip HS and Sebe, Nicu},\n  journal={IEEE Transactions on Neural Networks and Learning Systems (TNNLS)},\n  year={2021} \n}\n\n@inproceedings{tang2019attention,\n  title={Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation},\n  author={Tang, Hao and Xu, Dan and Sebe, Nicu and Yan, Yan},\n  booktitle={International Joint Conference on Neural Networks (IJCNN)},\n  year={2019}\n}\n```\n\n## Acknowledgments\nThis source code is inspired by [CycleGAN](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix), [GestureGAN](https://github.com/Ha0Tang/GestureGAN), and [SelectionGAN](https://github.com/Ha0Tang/SelectionGAN). \n\n## Contributions\nIf you have any questions/comments/bug reports, feel free to open a github issue or pull a request or e-mail to the author Hao Tang ([bjdxtanghao@gmail.com](bjdxtanghao@gmail.com)).\n\n## Collaborations\nI'm always interested in meeting new people and hearing about potential collaborations. If you'd like to work together or get in contact with me, please email bjdxtanghao@gmail.com. Some of our projects are listed [here](https://github.com/Ha0Tang).\n___\n*Figure out what you like. Try to become the best in the world of it.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHa0Tang%2FAttentionGAN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHa0Tang%2FAttentionGAN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHa0Tang%2FAttentionGAN/lists"}