https://github.com/dlpbc/gru-language-modeling
Language modeling using nairaland featured links as dataset
https://github.com/dlpbc/gru-language-modeling
gru language-modeling nairaland rnn theano
Last synced: about 1 month ago
JSON representation
Language modeling using nairaland featured links as dataset
- Host: GitHub
- URL: https://github.com/dlpbc/gru-language-modeling
- Owner: dlpbc
- License: mit
- Created: 2017-10-07T20:03:01.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2017-10-07T22:00:56.000Z (over 8 years ago)
- Last Synced: 2025-01-12T23:47:39.133Z (over 1 year ago)
- Topics: gru, language-modeling, nairaland, rnn, theano
- Language: Python
- Size: 8.75 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Language Modeling on Locally curated dataset
An interesting project that entails training neural net language model on a locally curated dataset. See below or [here](https://github.com/dlpbc/datasets/blob/master/README.md#nairaland-featured-links) for description of the dataset.
## Dataset - 5k featured links on nairaland
- 5000 sentences
- One sentence per line
- Each sentence is the title of a featured link thread on [nairaland](http://nairaland.com)
## Network Description
2 Layer Gated Recurrent Units (GRU). A slightly modified version of what was implemented [here](https://github.com/dennybritz/rnn-tutorial-gru-lstm).
## Network Configuration
- 500 units word embedding layer
- 400 gru units per gru layer
-vocabulary size: 500
- initial learning rate: 0.008 (annealed over epochs down to 0.00001)
- number of epochs: 115
## Network Performance
- 0.92 loss after 115 epochs (Pre-trained model located in **trained_model_param** directory.
## Usage
- Train Model
```
python train_gru_rnn.py
```
- Generate sentences (featured links) from pre-trained model
```
python sentence_generator.py
```
## Some Sample Sentences (featured links) Generated by trained model
- what to do during asuu strike
- nigeria vs cameroon : who is your man of the strike ?
- see new fayose for president 2019 today
- a nairalander 's pre-wedding photos
- how much do you members ways things ?
- do you has caught phone ?
- who is a his son ?
- quit to what youths to buhari , in they about
- president buhari receives state 2017
- 5 things you need to know when building a new home