Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/thisisiron/nmt-attention-tf2
👫 Effective Approaches to Attention-based Neural Machine Translation implemented as Tensorflow 2.0
https://github.com/thisisiron/nmt-attention-tf2
attention lstm natural-language-processing neural-machine-translation nlp nmt tensorflow tensorflow2 tf2 translation
Last synced: about 1 month ago
JSON representation
👫 Effective Approaches to Attention-based Neural Machine Translation implemented as Tensorflow 2.0
- Host: GitHub
- URL: https://github.com/thisisiron/nmt-attention-tf2
- Owner: thisisiron
- License: mit
- Created: 2019-04-18T06:33:50.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2019-10-07T14:26:40.000Z (about 5 years ago)
- Last Synced: 2024-10-27T18:58:00.576Z (about 2 months ago)
- Topics: attention, lstm, natural-language-processing, neural-machine-translation, nlp, nmt, tensorflow, tensorflow2, tf2, translation
- Language: Python
- Homepage:
- Size: 96.7 KB
- Stars: 36
- Watchers: 4
- Forks: 7
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-tensorflow-2 - Effective Approaches to Attention-based Neural Machine Translation
README
# nmt-attention-tf2
Effective Approaches to Attention-based Neural Machine Translation implemented as Tensorflow 2.0
## Requirements
Tensorflow 2.0
## Data
WMT'14 English-German data: https://nlp.stanford.edu/projects/nmt/
Download the datasets using the following script:
```
./download.sh
```
## Usage
```
usage: main.py [-h] [--mode MODE] [--config-path DIR] [--init-checkpoint FILE]
[--batch-size INT] [--epoch INT] [--embedding-dim INT]
[--max-len INT] [--units INT] [--dev-split REAL]
[--optimizer STRING] [--learning-rate REAL] [--dropout REAL]
[--method STRING]
train model from data
optional arguments:
-h, --help show this help message and exit
--mode MODE train or test
--config-path DIR config json path
--init-checkpoint FILE
checkpoint file
--batch-size INT batch size
--epoch INT epoch number
--embedding-dim INT embedding dimension
--max-len INT max length of a sentence
--units INT units
--dev-split REAL
--optimizer STRING optimizer
--learning-rate REAL learning rate
--dropout REAL dropout probability
--method STRING content-based function
```
Train command example
```
python main.py --max-len 50 --embedding-dim 100 --batch-size 60 --method concat
```
Test command example
```
python main.py --mode test --config-path training_checkpoints/{TRAINING_CHECKPOINT}/config.json
```
## Demo
I think this demo is poor performance because I don't have a large resource. So, The paper proposed embedding dimension sets 1000. But this demo's embedding dimension is 50. And this is trained only for 4 epochs.
If you don't have training_checkpoints directory, make training_checkpoints directory and proceed with the next step.
```
mkdir training_checkpoints
cd training_checkpoints
```
You can download [here](https://drive.google.com/open?id=19VtPQ-9gyLkNxRjD7GbjACaTM5xAz_lH). And you put DEMO directory in training_checkpoints directory.
```
python main.py --mode test --config-path training_checkpoints/DEMO/config.json
```
Example
```
Input Sentence or If you want to quit, type Enter Key : What is your name?
Early stopping
was ist ihr name ?
what is your name ?
```
## Results
| | Train Set BLEU | Test Set BLEU |
|---------|-------------------|---------------|
| Model | -- | -- |
## Reference
[Effective Approaches to Attention-based Neural Machine Translation](https://arxiv.org/abs/1508.04025?context=cs)