Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kimhc6028/forward-thinking-pytorch
Pytorch implementation of "Forward Thinking: Building and Training Neural Networks One Layer at a Time"
https://github.com/kimhc6028/forward-thinking-pytorch
Last synced: about 2 months ago
JSON representation
Pytorch implementation of "Forward Thinking: Building and Training Neural Networks One Layer at a Time"
- Host: GitHub
- URL: https://github.com/kimhc6028/forward-thinking-pytorch
- Owner: kimhc6028
- License: bsd-3-clause
- Created: 2017-06-13T11:31:25.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2017-06-14T00:48:29.000Z (almost 7 years ago)
- Last Synced: 2024-01-25T20:09:08.398Z (4 months ago)
- Language: Python
- Homepage: https://arxiv.org/pdf/1706.02480.pdf
- Size: 498 KB
- Stars: 65
- Watchers: 9
- Forks: 8
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- Awesome-pytorch-list - forward-thinking-pytorch
- Awesome-pytorch-list-CNVersion - forward-thinking-pytorch
README
# forward-thinking-pytorch
Pytorch implementation of [Forward Thinking: Building and Training Neural Networks One Layer at a Time](https://arxiv.org/pdf/1706.02480.pdf)
## Requirements
- Python 2.7
- [numpy](http://www.numpy.org/)
- [pytorch](http://pytorch.org/)
- [Matplotlib](https://matplotlib.org/users/installing.html)## Usage
$ ./run.sh
to run forward-thinking & forward-thinking (deep) & backpropagation & backpropagation (deep).
For forward-thinking experiment (5 layers),
$ python forward_thinking.py
If you want to do this with very deep model (20 layers),$ python forward_thinking.py --epoch 200 --deep
As a comparison, learn by backpropagation:
$ python backpropagate.py
Backpropagate with very deep model (20 layers),
$ python backpropagate.py --epoch 200 --deep
## Result
For 5 layers, forward-thinking learns slightly faster than backpropagation. Dips are observed when new layers are added.
When the model becomes very deep (20 layers), backpropagation cannot train the model. On the other hand, forward-thinking can train the model as usual.