https://github.com/jaanli/proximity_vi
This code accompanies the proximity variational inference paper.
https://github.com/jaanli/proximity_vi
Last synced: 3 months ago
JSON representation
This code accompanies the proximity variational inference paper.
- Host: GitHub
- URL: https://github.com/jaanli/proximity_vi
- Owner: jaanli
- License: mit
- Created: 2017-05-24T11:41:51.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2019-01-19T18:59:32.000Z (over 6 years ago)
- Last Synced: 2025-04-12T02:06:09.998Z (3 months ago)
- Language: Python
- Homepage: https://arxiv.org/abs/1705.08931
- Size: 850 KB
- Stars: 18
- Watchers: 2
- Forks: 10
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
### Proximity Variational Inference
This code accompanies the proximity variational inference paper: https://arxiv.org/abs/1705.08931
If you use this code, please cite us:
```
@article{altosaar2017proximity,
author={Altosaar, J and Ranganath, R and Blei, DM},
eprint={arXiv:1311.1704},
title={Proximity Variational Inference},
url={https://arxiv.org/abs/1705.08931},
year={2017}
}
```
The promise: Variational inference (left) is sensitive to initialization. Proximity variational inference (right) can help correct this.
### Data
Get the binarized MNIST dataset from [Hugo & Larochelle (2011)](http://proceedings.mlr.press/v15/larochelle11a.html), write it to `/tmp/binarized_mnist.hdf5`.
```
python get_binary_mnist.py
```
### Environment
I recommend anaconda: `brew cask install anaconda` on a mac, [bash installer](https://www.continuum.io/downloads) otherwise. To use the same environment:
```
conda env create -f environment.yml # may need to edit to choose between CPU or GPU version of tensorflow
source activate proximity_vi
```
The code assumes you have set the following environment variables. This enables easy switching between local and remote workstations.
```
> export DAT=/tmp
> export LOG=/tmp
```
### Sigmoid belief network experiment
This benchmarks proximity variational inference against deterministic annealing and vanilla variational inference, with good initialization and bad initialization (Tables 1 and 2 in the paper).
Each experiment takes about half a day on a Tesla P100 GPU:
```
./sigmoid_belief_network_grid.sh
# List final estimates of the ELBO and marginal likelihood
tail -n 1 $LOG/proximity_vi/*/*/*.log
# View training statistics on tensorboard
tensorboard --logdir $LOG/proximity_vi
```
### Variational autoencoder experiment
This tests the orthogonal proximity statistic to make optimization easier in a variational autoencoder. (Table 3 in the paper)
Each run takes a few minutes on a Tesla P100 GPU:
```
./deep_latent_gaussian_model_grid.sh
# List final estimates of the ELBO and marginal likelihood
tail -n 1 $LOG/proximity_vi/*/*/*.log
# View training statistics on tensorboard
tensorboard --logdir $LOG/proximity_vi
```
### Support
Please email me with any questions: [[email protected]](mailto:[email protected]).