https://github.com/stefanonardo/pytorch-esn
An Echo State Network module for PyTorch.
https://github.com/stefanonardo/pytorch-esn
deep-learning echo-state-networks esn machine-learning neural-networks python pytorch recurrent-neural-networks reservoir-computing
Last synced: about 1 year ago
JSON representation
An Echo State Network module for PyTorch.
- Host: GitHub
- URL: https://github.com/stefanonardo/pytorch-esn
- Owner: stefanonardo
- License: mit
- Created: 2018-03-25T22:28:33.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2023-02-17T15:07:31.000Z (over 3 years ago)
- Last Synced: 2024-11-12T12:49:27.116Z (over 1 year ago)
- Topics: deep-learning, echo-state-networks, esn, machine-learning, neural-networks, python, pytorch, recurrent-neural-networks, reservoir-computing
- Language: Python
- Homepage:
- Size: 302 KB
- Stars: 216
- Watchers: 9
- Forks: 44
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# PyTorch-ESN
PyTorch-ESN is a PyTorch module, written in Python, implementing Echo State Networks with leaky-integrated units. ESN's implementation with more than one layer is based on [DeepESN](https://arxiv.org/abs/1712.04323). The readout is trainable by ridge regression or by PyTorch's optimizers.
Its development started under my master thesis titled ["An Empirical Comparison of Recurrent Neural Networks on Sequence Modeling"](https://www.dropbox.com/s/gyt48dcktht7qur/document.pdf?dl=0), which was supervised by Prof. Alessio Micheli and Dr. Claudio Gallicchio at the University of Pisa.
## Prerequisites
* PyTorch
## Basic Usage
### Offline training (ridge regression)
#### SVD
Mini-batch mode is not allowed with this method.
```python
from torchesn.nn import ESN
from torchesn.utils import prepare_target
# prepare target matrix for offline training
flat_target = prepare_target(target, seq_lengths, washout)
model = ESN(input_size, hidden_size, output_size)
# train
model(input, washout, hidden, flat_target)
# inference
output, hidden = model(input, washout, hidden)
```
#### Cholesky or inverse
```python
from torchesn.nn import ESN
from torchesn.utils import prepare_target
# prepare target matrix for offline training
flat_target = prepare_target(target, seq_lengths, washout)
model = ESN(input_size, hidden_size, output_size, readout_training='cholesky')
# accumulate matrices for ridge regression
for batch in batch_iter:
model(batch, washout[batch], hidden, flat_target)
# train
model.fit()
# inference
output, hidden = model(input, washout, hidden)
```
#### Classification tasks
For classification, just use one of the previous methods and pass 'mean' or
'last' to ```output_steps``` argument.
```python
model = ESN(input_size, hidden_size, output_size, output_steps='mean')
```
For more information see docstrings or section 4.7 of "A Practical Guide to Applying
Echo State Networks" by Mantas Lukoševičius.
### Online training (PyTorch optimizer)
Same as PyTorch.