https://github.com/oxford-cs-ml-2015/practical6

Practical 6: LSTM language models
https://github.com/oxford-cs-ml-2015/practical6

Last synced: about 1 month ago
JSON representation

Practical 6: LSTM language models

• Host: GitHub
• URL: https://github.com/oxford-cs-ml-2015/practical6
• Owner: oxford-cs-ml-2015
• Created: 2015-03-10T23:32:01.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2015-06-09T13:29:49.000Z (over 9 years ago)
• Last Synced: 2024-08-03T01:14:14.182Z (4 months ago)
• Language: Lua
• Homepage: https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/
• Size: 347 KB
• Stars: 260
• Watchers: 30
• Forks: 84
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-neat-rnn - [Oxford practical 6

README

        
# Practical 6

Machine Learning, spring 2015

In this practical, we train an LSTM for character-level language modelling. Since this is the last week for practicals, it will be **extremely short** and does not require writing code, and is due by the end of the Friday's session (regardless of whether you are from the Wednesday or Friday session).

See PDF for details.

## Setup

Setup will be the same as last time in practical 1. Please refer to the [practical 1 repository](https://github.com/oxford-cs-ml-2015/practical1), and run the script as instructed last time. If you get an error that `nngraph` is not installed, run:

```

luarocks install nngraph

```

# Do this before reading the pdf

Clone the practical **and** download the associated data:

```

git clone https://github.com/oxford-cs-ml-2015/practical6.git

cd practical6

wget http://www.cs.ox.ac.uk/people/brendan.shillingford/teaching/practical6-data.tar.gz

tar xvf practical6-data.tar.gz

```

and start training the model:

```

th train.lua -vocabfile vocab.t7 -datafile train.t7 

```

**Make note of** the time at which you run the `train.lua` script. Every several iterations, the training script will save the current model (including its parameters) to a file called `model_autosave.t7`. You can make snapshots of this file if you want, but this is not required for the practical.

# For users outside of Oxford's CS lab

The `practical6-data.tar.gz` file is for 64-bit little-endian CPUs. For all other machines (i.e. if running `uname -m` doesn't print out `x86_64`), then see this comment for instructions:

. This is the same data, but using ASCII serialization.

You may also want to use this faster LSTM factory method, instead of the one in this repository:  which performs all the matrix multiplications at once followed by several `nn.Narrow` operations to extract out the gate values; read its comments for details.

# See course page for practicals