Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/anujdutt9/language-model-using-rnn
Text Generation using basic RNN from scratch using Python and Numpy.
https://github.com/anujdutt9/language-model-using-rnn
language-model numpy python rnn text-generation
Last synced: about 1 month ago
JSON representation
Text Generation using basic RNN from scratch using Python and Numpy.
- Host: GitHub
- URL: https://github.com/anujdutt9/language-model-using-rnn
- Owner: anujdutt9
- Created: 2017-04-14T21:35:49.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2017-04-14T21:45:27.000Z (almost 8 years ago)
- Last Synced: 2024-11-13T03:34:50.086Z (3 months ago)
- Topics: language-model, numpy, python, rnn, text-generation
- Language: Python
- Homepage:
- Size: 22.9 MB
- Stars: 0
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# RNN Language Model
***This repository contains the code for Language Model for Text Generation using RNN from scratch using Python and Numpy.***
*This code implements a basic Recurrent Neural Network for a Language Model that predicts the next word in a sentence and forms a final sentence using proper punctuations.*
**Note:** Use this code to understand the underlaying concept from scratch that how a Language Model works using a RNN with Back Propagation through Time. Running of this code on the PC/Laptop is not advisable as it might take days or weeks to train and generate the text as the data size is large.
# Requirements
1. **Python 3 and above**
2. **Numpy**
# Usage
```
python3 rnn_lm.py
```
# Result
*The following image shows the working of the code. This image shows the running of code on a small data of 1000 words from the main dataset. We can see that the loss decreases with each epoch. This proves that the code works fine and given the time and compute power, it can generate the text.*
* Some generated text samples
```
Anyway, to the city scene you’re an idiot teenager.
What ? ! ! ! ! ignore!
Screw fitness, you’re saying: https
Thanks for the advice to keep my thoughts around girls.
Yep, please disappear with the terrible generation.
```
# Resources
| S.No. | Papers / Blogs / Authors | Paper Links |
| ------ | --------------------------------------------------------- | ---------------------------------------------------- |
|1. | Andrej Karpathy's Blog "The Unreasonable Effectiveness of Recurrent Neural Networks" | http://karpathy.github.io/2015/05/21/rnn-effectiveness/ |
|2. | Denny Britz's Blog [WILDML] |http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ |
|3. | Hugo Larochelle's Videos on NLP | https://www.youtube.com/watch?v=OzZIOiMVUyM&list=PLSaRkipE5TJhNGgRrpRL180SZ1Mg-2jxT |