https://github.com/ketranm/RMN
Recurrent Memory Network - Torch implementation
https://github.com/ketranm/RMN
Last synced: 20 days ago
JSON representation
Recurrent Memory Network - Torch implementation
- Host: GitHub
- URL: https://github.com/ketranm/RMN
- Owner: ketranm
- Created: 2016-03-07T12:15:18.000Z (about 10 years ago)
- Default Branch: master
- Last Pushed: 2016-10-25T08:17:48.000Z (over 9 years ago)
- Last Synced: 2026-02-08T08:15:25.171Z (about 1 month ago)
- Language: Lua
- Homepage:
- Size: 238 MB
- Stars: 35
- Watchers: 3
- Forks: 8
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome-Code - Recurrent Memory Network for Language Modeling
README
# Recurrent Memory Networks
This code implements Recurrent Memory Networks (RM and RMR) described in
Recurrent Memory Networks for Language Modeling
Ke Tran, Arianna Bisazza, and Christof Monz
In Proceedings of NAACL 2016
Much of this code is based on [char-nn](https://github.com/karpathy/char-rnn).
## Getting data
The data is pushed using [git-lfs](https://git-lfs.github.com/). To get the data, you need to have git-lfs installed, then just do
```
$ git lfs pull
```
## Use the code
Here is an example how to run the code.
```
$ th RM.lua -max_seq_length 80 -min_seq_length 10 -max_epochs 20 \
-data_dir data/it -print_every 200 -num_layers 1 -mem_size 15 \
-learning_rate 1 -emb_size 128 -rnn_size 128 -nhop 1 \
-checkpoint_dir checkpoint
```
There are many settings we didn't perform in our paper, for example increasing the embedding size, stacking multiple LSTM layers, etc. The reason is that we have a limitted time to spend on shared GPUs.
## Data Processing
Since we are interested mostly in long distant dependencies, we only selected sentences whose length is in between 10 and 80 tokens (use this option `-max_seq_length 80 -min_seq_length 10`). If you want to compare to our results, ideally, you should use the same setting.
## TODO:
- clean up the code (remove some experimental settings)
- fast output layer?