https://github.com/chunyuanli/psgld

AAAI & CVPR 2016: Preconditioned Stochastic Gradient Langevin Dynamics (pSGLD)
https://github.com/chunyuanli/psgld

mcmc preconditioner sampling-methods stochastic-gradient-descent

Last synced: about 1 year ago
JSON representation

AAAI & CVPR 2016: Preconditioned Stochastic Gradient Langevin Dynamics (pSGLD)

• Host: GitHub
• URL: https://github.com/chunyuanli/psgld
• Owner: ChunyuanLI
• Created: 2016-02-28T20:34:31.000Z (over 10 years ago)
• Default Branch: master
• Last Pushed: 2018-09-16T02:02:58.000Z (over 7 years ago)
• Last Synced: 2025-02-03T00:54:34.335Z (over 1 year ago)
• Topics: mcmc, preconditioner, sampling-methods, stochastic-gradient-descent
• Language: Matlab
• Homepage:
• Size: 27.4 MB
• Stars: 35
• Watchers: 2
• Forks: 12
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# pSGLD

Preconditioned Stochastic Gradient Langevin Dynamics (pSGLD)

Links:

[Implementation on TensorFlow Website](https://www.tensorflow.org/probability/api_docs/python/tfp/optimizer/StochasticGradientLangevinDynamics)

## Simulation (2D Gaussian Example in Figure 1 of the paper)

- Simulation 1 provides _Average Absolute Error of Sample Covariance_ vs _AutoCorrelation Time (ACT)_

- Simulation 2 provides first 600 samples from SGLD and pSGLD



## Experiments on Deep Neural Networks (Keep updating)

- Start to run 'test_FNN_mnist.m' to test a 2-layer FNN with 400 hidden units each . 

- You may also modify line 'linSizes  = [400 400 data.outSize]' to other configurations. 

## Citation

Please cite our AAAI paper if it helps your research:

	@inproceedings{pSGLD_AAAI2016,

	  title={Preconditioned stochastic gradient Langevin dynamics for deep neural networks},

	  author={Li, Chunyuan and Chen, Changyou and Carlson, David and Carin, Lawrence},

	  booktitle={AAAI},

	  Year  = {2016}

	}