https://github.com/zphang/num2num
PyTorch RNNs on a toy example of translating between numbers in words and digits
https://github.com/zphang/num2num
Last synced: 3 months ago
JSON representation
PyTorch RNNs on a toy example of translating between numbers in words and digits
- Host: GitHub
- URL: https://github.com/zphang/num2num
- Owner: zphang
- Created: 2017-10-19T21:15:59.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2017-10-22T15:13:44.000Z (over 7 years ago)
- Last Synced: 2025-01-29T13:43:33.235Z (4 months ago)
- Language: Python
- Size: 24.8 MB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Home
An example (Attentional) Encoder-Decoder RNN in PyTorch, applied to a toy data set: translating back and forth between numbers-as-text ("one thousand and two") and numbers-as-digits ("1002").
### Introduction
This project began as an attempt to code up an implementation of attentional RNNs against a simple data set, to understand the inner-workings of the attention mechanism. It turns out that the implementation can get fairly non-trivial.
In particular, the attention portions of the code heavily referenced the implementation in https://github.com/spro/practical-pytorch/tree/master/seq2seq-translation.
*Attention plot between input (top) and output (left). The attention isn't perfect because our models can be way over-parameterized compared the the problem.*
### Requirements
- Python 3.6
- [PyTorch 0.2.0](http://pytorch.org/)
- (For generating data) `num2words`, `tqdm`
- `pandas`, `argparse`, `matplotlib`, `seaborn`
### Data
Using `num2words`, we can generate random numbers and get the "ground-truth" text versions of those numbers. Commas are removed, and the remaining tokens have been pre-computed and saved in (`datafiles/word2num_tokens`)[datafiles/word2num_tokens].
Note that `num2num` supports both word-level and character-level modeling, so both tokenizations have been pre-computed.
### Usage
1. Git clone this repository.
2. (Optional) Generate training and validation datasets. A very small sample training and validation dataset is included with the project.
* E.g.
```bash
python gendata.py \
--output=datafiles/train_numbers.csv \
--size=100000
python gendata.py \
--output=datafiles/val_numbers.csv \
--size=10000
```
* Run `python gendata.py -h` or see `gendata.py` for details and more options.
3. Train the model:
* E.g.
```bash
python run_train.py \
--train_data_path=datafiles/train_numbers.csv \
--val_data_path=datafiles/val_numbers.csv \
--plot_attn_show=False \
--plot_attn_save_path="output/attn_plots" \
--model_save_path="output/models
```
* Run `python run_train.py -h` or see `orchestrate.py` and `num2num/config.py` for details and more options
4. Sample from model:
* E.g.
```bash
python run_sample.py \
--val_data_path=datafiles/val_numbers.csv \
--model_save_path="output/models/my_favorite_model
```
* Run `python run_sample.py -h` or see `orchestrate.py` and `num2num/config.py` for details and more options
### Road map
- [ ] Documentation
- [ ] Tests