Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/woozzu/dong_iccv_2017

A PyTorch implementation of the paper "Semantic Image Synthesis via Adversarial Learning" in ICCV 2017
https://github.com/woozzu/dong_iccv_2017

Last synced: 2 months ago
JSON representation

A PyTorch implementation of the paper "Semantic Image Synthesis via Adversarial Learning" in ICCV 2017

Host: GitHub
URL: https://github.com/woozzu/dong_iccv_2017
Owner: woozzu
License: mit
Created: 2017-08-22T06:49:52.000Z (almost 7 years ago)
Default Branch: master
Last Pushed: 2017-08-24T08:48:48.000Z (almost 7 years ago)
Last Synced: 2024-01-27T03:18:46.570Z (5 months ago)
Language: Python
Homepage:
Size: 2.22 MB
Stars: 146
Watchers: 9
Forks: 25
Open Issues: 8
Metadata Files:
- Readme: README.md
- License: LICENSE

Lists

Awesome-pytorch-list - Semantic Image Synthesis via Adversarial Learning
Awesome-pytorch-list-CNVersion - Semantic Image Synthesis via Adversarial Learning

README

        # Semantic Image Synthesis via Adversarial Learning

This is a PyTorch implementation of the paper [Semantic Image Synthesis via Adversarial Learning](https://arxiv.org/abs/1707.06873).

![Model architecture](images/architecture.png)

## Requirements

- [PyTorch](https://github.com/pytorch/pytorch) 0.2

- [Torchvision](https://github.com/pytorch/vision)

- [Pillow](https://pillow.readthedocs.io/en/4.2.x/)

- [fastText.py](https://github.com/salestock/fastText.py) (Note: if you have a problem when loading a pretrained model, try [my fixed code](https://github.com/woozzu/fastText.py/tree/feature/udpate-fasttext-to-f24a781-fix))

- [NLTK](http://www.nltk.org)

## Pretrained word vectors for fastText

Download a pretrained [English](https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.en.zip) word vectors. You can see the list of pretrained vectors on [this page](https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md).

## Datasets

- Oxford-102 flowers: [images](http://www.robots.ox.ac.uk/~vgg/data/flowers/102) and [captions](https://drive.google.com/file/d/0B0ywwgffWnLLMl9uOU91MV80cVU/view?usp=sharing)

- Caltech-200 birds: [images](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html) and [captions](https://drive.google.com/file/d/0B0ywwgffWnLLLUc2WHYzM0Q2eWc/view?usp=sharing)

The caption data is from [this repository](https://github.com/reedscot/icml2016). After downloading, modify `CONFIG` file so that all paths of the datasets point to the data you downloaded.

## Run

- `scripts/train_text_embedding_[birds/flowers].sh`  

Train a visual-semantic embedding model using the method of [Kiros et al.](https://arxiv.org/abs/1411.2539).

- `scripts/train_[birds/flowers].sh`  

Train a GAN using a pretrained text embedding model.

- `scripts/test_[birds/flowers].sh`  

Generate some examples using original images and semantically relevant texts.

## Results

![Flowers](images/results_flowers.png)

![Birds](images/results_birds.png)

## Acknowledgements

- [Text to image synthesis](https://github.com/reedscot/icml2016)

- [StackGAN](https://github.com/hanzhanggit/StackGAN)

We would like to thank Hao Dong, who is one of the first authors of the paper [Semantic Image Synthesis via Adversarial Learning](https://arxiv.org/abs/1707.06873), for providing helpful advice for the implementation.