https://github.com/jhhuang96/ConvLSTM-PyTorch

ConvLSTM/ConvGRU (Encoder-Decoder) with PyTorch on Moving-MNIST
https://github.com/jhhuang96/ConvLSTM-PyTorch

convgru convlstm encoder-decoder gru lstm pytorch-implementation rnn spatio-temporal time-series

Last synced: about 1 month ago
JSON representation

ConvLSTM/ConvGRU (Encoder-Decoder) with PyTorch on Moving-MNIST

• Host: GitHub
• URL: https://github.com/jhhuang96/ConvLSTM-PyTorch
• Owner: jhhuang96
• License: mit
• Created: 2019-08-13T08:10:37.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2022-10-13T02:04:37.000Z (about 2 years ago)
• Last Synced: 2024-08-01T19:45:46.864Z (4 months ago)
• Topics: convgru, convlstm, encoder-decoder, gru, lstm, pytorch-implementation, rnn, spatio-temporal, time-series
• Language: Python
• Homepage:
• Size: 9.33 MB
• Stars: 371
• Watchers: 8
• Forks: 83
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-cv - PyTorch Implementation of ConvLSTM Cell
• awesome-cv - PyTorch Implementation of ConvLSTM Cell

README

        


# ConvLSTM-Pytorch

## ConvRNN cell

Implement ConvLSTM/ConvGRU cell with Pytorch. This idea has been proposed in this paper: [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://arxiv.org/abs/1506.04214)

## Experiments with ConvLSTM on MovingMNIST

Encoder-decoder structure. Takes in a sequence of 10 movingMNIST fames and attempts to output the remaining frames.

## Instructions

Requires `Pytorch v1.1` or later (and GPUs)

Clone repository

```

git clone https://github.com/jhhuang96/ConvLSTM-PyTorch.git

```

To run endoder-decoder network for prediction moving-mnist:

```python

python main.py

```

## Moving Mnist Generator

The script ``data/mm.py`` is the script to generate customized Moving Mnist based on [MNIST](http://yann.lecun.com/exdb/mnist/). 

```python

MovingMNIST(is_train=True,

            root='data/',

            n_frames_input=args.frames_input,

            n_frames_output=args.frames_output,

            num_objects=[3])

```

- is_train: If True, use script to generate data. If False, directly use Moving Mnist data  downloaded from http://www.cs.toronto.edu/~nitish/unsupervised_video/

- root: The path of MNIST data

- n_frames_input: Number of input frames (int)

- n_frames_output: Number of output frames (int)

- num_objects:  Number of digits in a frame (List) . [3] means there are 3 digits in each frame

## Result

 ![Result](https://github.com/jhhuang96/ConvLSTM-PyTorch/tree/master/images/movingmnist.png)

- The first line is the real data for the first 10 frames

- The second line is prediction of the model for the last 10 frames

## Citation

```

@inproceedings{xingjian2015convolutional,

  title={Convolutional LSTM network: A machine learning approach for precipitation nowcasting},

  author={Xingjian, SHI and Chen, Zhourong and Wang, Hao and Yeung, Dit-Yan and Wong, Wai-Kin and Woo, Wang-chun},

  booktitle={Advances in neural information processing systems},

  pages={802--810},

  year={2015}

}

@inproceedings{xingjian2017deep,

    title={Deep learning for precipitation nowcasting: a benchmark and a new model},

    author={Shi, Xingjian and Gao, Zhihan and Lausen, Leonard and Wang, Hao and Yeung, Dit-Yan and Wong, Wai-kin and Woo, Wang-chun},

    booktitle={Advances in Neural Information Processing Systems},

    year={2017}

}

```