https://github.com/revsic/numpy-rnn
numpy implementation of Recurrent Neural Network
https://github.com/revsic/numpy-rnn
Last synced: 14 days ago
JSON representation
numpy implementation of Recurrent Neural Network
- Host: GitHub
- URL: https://github.com/revsic/numpy-rnn
- Owner: revsic
- License: mit
- Created: 2017-06-20T08:15:28.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2019-06-02T09:43:31.000Z (almost 6 years ago)
- Last Synced: 2025-05-05T17:31:53.943Z (14 days ago)
- Language: Jupyter Notebook
- Size: 223 KB
- Stars: 41
- Watchers: 2
- Forks: 10
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# numpy-RNN
numpy implementation of Recurrent Neural Networks
- [RNN_numpy](./RNN_numpy.ipynb) based on [iamtrask's github.io](https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/)
- [LSTM_numpy](./LSTM_numpy.ipynb) based on [wiseodd's github.io](http://wiseodd.github.io/techblog/2016/08/12/lstm-backprop/)
## Recurrent Neural Networks
Training Vanilla RNN for binary addition.
```python
largest = pow(2, BIN_DIM)
decimal = np.array([range(largest)]).astype(np.uint8).T
binary = np.unpackbits(decimal, axis=1)
```
Simply prepare the binary digit database. And propagate forward and backward to train binary addition.
```python
hidden = sigmoid(np.dot(X, w0) + np.dot(hidden_values[-1], wh))
output = sigmoid(np.dot(hidden, w1))
pred[pos] = np.round(output[0][0])
```
Training was successful with 100% accuracy.
```
----------
Iter 9000
Error : [ 0.68126419]
Pred : [1 0 1 0 1 0 0 0]
True : [1 0 1 0 1 0 0 0]
108 + 60 = 168
----------
```
## LSTM Networks
Training LSTM RNN for mnist handwritten digit recognition.
```python
mnist = tf.keras.datasets.mnist
(train_x, train_y), _ = mnist.load_data()
```
Prepare the mnist digit database with `tf.keras.datasets.mnist`.
```python
caches, states = LSTM_Cell(Xt)
c, h = states[-1]
out = np.dot(h, wy) + by
pred = softmax(out)
entropy = cross_entropy(pred, Y)
```
Train the LSTM model with BPTT and it was successful to recognize handwritten digits.
