https://github.com/lrjconan/RBP

Recurrent Back Propagation, Back Propagation Through Optimization, ICML 2018
https://github.com/lrjconan/RBP

icml-2018 optimization pytorch rbp

Last synced: about 2 months ago
JSON representation

Recurrent Back Propagation, Back Propagation Through Optimization, ICML 2018

• Host: GitHub
• URL: https://github.com/lrjconan/RBP
• Owner: lrjconan
• License: mit
• Created: 2018-07-09T18:16:31.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2019-01-12T02:09:58.000Z (over 5 years ago)
• Last Synced: 2024-07-29T07:32:14.568Z (about 2 months ago)
• Topics: icml-2018, optimization, pytorch, rbp
• Language: Python
• Homepage: https://arxiv.org/abs/1803.06396
• Size: 4.75 MB
• Stars: 39
• Watchers: 3
• Forks: 4
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# RBP

This is the PyTorch implementation of [Recurrent Back Propagation](https://arxiv.org/pdf/1803.06396.pdf) as described in the following ICML 2018 paper:

```

@article{liao2018reviving,

  title={Reviving and Improving Recurrent Back-Propagation},

  author={Liao, Renjie and Xiong, Yuwen and Fetaya, Ethan and Zhang, Lisa and Yoon, KiJung and Pitkow, Xaq and Urtasun, Raquel and Zemel, Richard},

  journal={arXiv preprint arXiv:1803.06396},

  year={2018}

}

```

## Setup

To set up experiments, we need to build our customized operators by running the following scripts:

```

./setup.sh

``` 

## Dependencies

Python 3, PyTorch(0.4.0)

## Run Demos

* To run experiments ```X``` where ```X``` is one of {```hopfield```, ```cora```, ```pubmed```, ```hypergrad```}:

  ```python run_exp.py -c config/X.yaml```

  

**Notes**:

* Most hyperparameters in the configuration yaml file are self-explanatory.

* To switch between BPTT, TBPTT and RBP variants, you need to specify ```grad_method``` in the config file.

* Conjugate gradient based RBP requires support of forward mode auto-differentiation which we only provided for the experiments of Hopfield networks and graph neural networks (GNNs). You can check the comments in ```model/rbp.py``` for more details.

## Cite

Please cite our paper if you use this code in your research work.

## Questions/Bugs

Please submit a Github issue or contact [email protected] if you have any questions or find any bugs.