Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/a-jacobson/minimal-nmt
A minimal nmt example to serve as an seq2seq+attention reference.
https://github.com/a-jacobson/minimal-nmt
attention-mechanism beam-search neural-machine-translation python-3-6 pytorch seq2seq tutorial
Last synced: about 8 hours ago
JSON representation
A minimal nmt example to serve as an seq2seq+attention reference.
- Host: GitHub
- URL: https://github.com/a-jacobson/minimal-nmt
- Owner: A-Jacobson
- License: mit
- Created: 2018-03-10T04:04:04.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2019-09-23T19:07:40.000Z (over 5 years ago)
- Last Synced: 2025-01-18T09:45:20.973Z (7 days ago)
- Topics: attention-mechanism, beam-search, neural-machine-translation, python-3-6, pytorch, seq2seq, tutorial
- Language: Jupyter Notebook
- Homepage:
- Size: 235 KB
- Stars: 36
- Watchers: 3
- Forks: 6
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Minimal Neural Machine Translation
## Resources
NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE
https://arxiv.org/pdf/1409.0473.pdf
Effective Approaches to Attention-based Neural Machine Translation
https://arxiv.org/pdf/1508.04025.pdf
Massive Exploration of Neural Machine Translation Architectures
https://arxiv.org/pdf/1703.03906.pdf
## Contents
- Encoder --> Attention --> Decoder Architecture.
- Luong Attention.
- Training on Multi30k German to English translation task.
- Attention Visualization.
- Teacher Forcing.
- Greedy Decoding.
- [nmt tutorial notebook](https://github.com/A-Jacobson/minimal-nmt/blob/master/nmt_tutorial.ipynb)
- minimal beam search decoding.
## Setup
1. install pytorch 0.4:
```
conda install pytorch -c pytorch=0.4.1
```
2. install other requirements:
```
pip install -r requirements.txt
```
## System Requirements
Training with a batch size of 32 takes ~3gb GPU ram.
If this is too much, lower the batch size or reduce network dimensionality in `hyperparams.py`.
## Usage
```
python train.py
```
view logs in Tensorboard
decent alignments should be seen after 2-3 epochs.
```
tensorboard --logdir runs
```
(partially trained attention heatmap)
