Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mishig25/synthetic-gradients-keras
Keras implementation of `Decoupled Neural Interfaces using Synthetic Gradients`
https://github.com/mishig25/synthetic-gradients-keras
Last synced: about 1 month ago
JSON representation
Keras implementation of `Decoupled Neural Interfaces using Synthetic Gradients`
- Host: GitHub
- URL: https://github.com/mishig25/synthetic-gradients-keras
- Owner: mishig25
- Created: 2018-08-23T17:03:23.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-10-19T01:07:28.000Z (about 6 years ago)
- Last Synced: 2024-11-18T20:13:22.564Z (about 2 months ago)
- Language: Python
- Size: 9.77 KB
- Stars: 12
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Decoupled Neural Interfaces using Synthetic Gradients
Keras as an interface to Tensorflow implementation of Decoupled Neural Interfaces using Synthetic Gradients.
Link to the paper: [https://arxiv.org/abs/1608.05343](https://arxiv.org/abs/1608.05343)
GIF demonstrating decoupled learning through synthetic gradients. Source: [DeepMind blog post](https://deepmind.com/blog/decoupled-neural-networks-using-synthetic-gradients/) by Max Jaderberg.
### Contents:
- `main.py` - main function
- `model.py` - synthetic grads implementation
- `demo_nb.ipynb` - jupyter notebook for demonstrating contents and usage of `model.py`
### Prerequisites:
- Python 3.6
- Keras 2.2.0
- Tensorflow 1.8.0
### Usage:
First option:
```
main.py [-h] [-I ITERATIONS] [-B BATCH] [-P UPDATE_PROB] [-L L_RATE]
optional arguments:
-h, --help show this help message and exit
-I ITERATIONS, --iterations ITERATIONS
Number of Iterations: int
-B BATCH, --batch BATCH
Batch Size: int
-P UPDATE_PROB, --update_prob UPDATE_PROB
Synthetic Grad Update Probability: float [0,1]
-L L_RATE, --l_rate L_RATE
Learning Rate: float
```
Second option:
```
Use Jupyter Lab or Notebooks to open `demo_nb.ipynb`
```
### Tested on:
- OS: ubuntu 16.04 LTS
- GPU: single GeForce GTX 1070
### Results
| | Accuracy | Loss |
|------|-------|-------|
|MNIST| 0.917 | 0.288 |
### References
- [Guide by Andrew Trask](https://iamtrask.github.io/2017/03/21/synthetic-gradients/)
- [Tensorflow Implementation](https://github.com/nitarshan/decoupled-neural-interfaces)
- [PyTorch Implementation](https://github.com/andrewliao11/dni.pytorch)