https://github.com/jiasenlu/visdial.pytorch
visual dialog model in pytorch
https://github.com/jiasenlu/visdial.pytorch
Last synced: 6 months ago
JSON representation
visual dialog model in pytorch
- Host: GitHub
- URL: https://github.com/jiasenlu/visdial.pytorch
- Owner: jiasenlu
- License: mit
- Created: 2017-10-04T23:55:13.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2018-05-16T08:13:02.000Z (over 7 years ago)
- Last Synced: 2024-05-22T19:36:02.400Z (over 1 year ago)
- Language: Python
- Size: 38.1 KB
- Stars: 110
- Watchers: 7
- Forks: 43
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# visDial.pytorch
Visual Dialog model in pytorch
### Introduction
This is the pytorch implementation of our NIPS 2017 paper ["Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model"](https://arxiv.org/abs/1706.01554)
### Disclaimer
This is the reimplementation code of visual dialog model based on Pytorch. Our original code was implemented during the first author's internship. All the results presented in our paper were obtained based on the original code, which cannot be released since the firm restriction. This project is an attempt to reproduce the results in our paper.
### Citation
If you find this code useful, please cite the following paper:
@article{lu2017best,
title={Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model},
author={Lu, Jiasen and Kannan, Anitha and and Yang, Jianwei and Parikh, Devi and Batra, Dhruv},
journal={NIPS},
year={2017}
}
### Dependencies
1. PyTorch. Install [PyTorch](http://pytorch.org/) with proper commands. Make sure you also install *torchvision*.
### Evaluation
* The preprocessed feature can be found [here](https://filebox.ece.vt.edu/~jiasenlu/codeRelease/visDial.pytorch/data/)
* The pre-trained model can be found [here](https://filebox.ece.vt.edu/~jiasenlu/codeRelease/visDial.pytorch/model/)
To evaluate the pre-trained model on validation set, first use the script to download the feature and pre-trained model.
```
python script/download.py --path [path_to_download]
```
After download the feature and pre-trained model, you can run the evaluation script by using following command
* Evaluate the discriminative model:
```
python eval/eval_D.py --data_dir [path_to_root] --model_path [path_to_root]/save/HCIAE-D-MLE.pth --cuda
```
* Evaluate the MLE trained generative model:
```
python eval/eval_G.py --data_dir [path_to_root] --model_path [path_to_root]/save/HCIAE-G-MLE.pth --cuda
```
* Evaluate the DIS trained generative model:
```
python eval/eval_G_DIS.py --data_dir [path_to_root] --model_path [path_to_root]/save/HCIAE-G-DIS.pth --cuda
```
You will get the similar results as in the paper :)
### Train a visual dialog model.
#### Preparation
First download the feature. from [here](https://filebox.ece.vt.edu/~jiasenlu/codeRelease/visDial.pytorch/model/)
#### Training
* Train the discriminative model:
```
python train/train_D.py --cuda
```
* Train the MLE trained generative model:
```
python train/train_G.py --cuda
```
* Train the DIS trained generative model:
First, train or download the pretrained discriminative model or generative model, and put it under save
```
python train/train_all.py --cuda --update LM
```