https://github.com/pfnet-research/vat_nmt
Implementation of "Effective Adversarial Regularization for Neural Machine Translation", ACL 2019
https://github.com/pfnet-research/vat_nmt
acl2019 adversarial neural-machine-translation nlp nmt vat
Last synced: 15 days ago
JSON representation
Implementation of "Effective Adversarial Regularization for Neural Machine Translation", ACL 2019
- Host: GitHub
- URL: https://github.com/pfnet-research/vat_nmt
- Owner: pfnet-research
- License: mit
- Created: 2019-07-26T08:40:08.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2020-01-11T07:11:46.000Z (almost 6 years ago)
- Last Synced: 2025-04-13T10:40:36.549Z (6 months ago)
- Topics: acl2019, adversarial, neural-machine-translation, nlp, nmt, vat
- Language: Python
- Size: 37.1 KB
- Stars: 21
- Watchers: 2
- Forks: 6
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Virtual Adversarial Training for NMT (Transformer model)
Implementation of "Effective Adversarial Regularization for Neural Machine Translation", ACL 2019
## References
Motoki Sato, Jun Suzuki, Shun Kiyono. "Effective Adversarial Regularization for Neural Machine Translation", ACL 2019
[paper](https://www.aclweb.org/anthology/P19-1020) [bib](https://aclweb.org/anthology/papers/P/P19/P19-1020.bib)
# How to use
## Requirements
- Python3.6+
- Chainer 6.x+
- Cupy 6.x+
```
# install chainer and cupy
$ pip install cupy
$ pip install chainer
$ pip install logzero
```
Please see how to install chainer: https://docs.chainer.org/en/stable/install.html
## Train (iwslt2016-de-en)
```
$ python3 -u chainer_transformer.py --mode train --gpus 0 --dataset iwslt2016-de-en --seed 1212 --epoch 40 --out model_transformer_de-en
```
## Train with VAT (iwslt2016-de-en)
```
$ python3 -u chainer_transformer.py --mode train --gpus 0 --dataset iwslt2016-de-en --seed 1212 --epoch 40 --out model_transformer_de-en_vat_enc --use-vat 1 --eps 1.0 --perturbation-target 0
```
### perturbation types
| perturbation-target | (enc, dec, enc-dec) |
----|----
| 0 | enc |
| 1 | dec |
| 0 1 | enc-dec (both) |
### VAT, Adv, VAT-Adv
| use-vat | (vat, adv, vat-adv) |
----|----
| 0 | non (baseline) |
| 1 | vat |
| 2 | adv |
| 3 | vat-adv (both) |
## Eval
```
$ python3 -u chainer_transformer.py --mode test --gpus 0 --dataset iwslt2016-de-en --batchsize 600 --model model_transformer_de-en/model_epoch_40.npz --beam 20 --max-length 60 --datatype eval1
```
# License
MIT License. Please see the LICENSE file for details.
# Authors
We thank Takeru Miyato ([@takerum](https://github.com/takerum)), who gave us valuable comments about AdvT/VAT.
The codebase of the transformer is developed by Shun Kiyono ([@butsugiri](https://github.com/butsugiri))
# Contact
Please give me comments or questions: [@aonotas](https://github.com/aonotas)