https://github.com/tkusmierczyk/machine_learning_demos
Notebooks and code snippets demonstrating machine learning techniques.
https://github.com/tkusmierczyk/machine_learning_demos
bayesian-optimization global-optimization machine-learning
Last synced: 9 months ago
JSON representation
Notebooks and code snippets demonstrating machine learning techniques.
- Host: GitHub
- URL: https://github.com/tkusmierczyk/machine_learning_demos
- Owner: tkusmierczyk
- Created: 2019-06-18T13:48:17.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2024-10-29T14:27:58.000Z (over 1 year ago)
- Last Synced: 2024-10-29T17:11:30.107Z (over 1 year ago)
- Topics: bayesian-optimization, global-optimization, machine-learning
- Language: Jupyter Notebook
- Size: 5.57 MB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Machine Learning Demos
Notebooks and code snippets demonstrating various machine learning techniques:
9. [Three approaches for computing trace of hessian for i-th output of a PyTorch module](trace_of_hessian_for_model_output.ipynb)
* Hessian computation using autograd
* Hessian computation using Laplace library
* Trace of Hessian computation using the Hutchinson's estimator
8. [PyTorch implementation of SNGP (Spectral-normalized Neural Gaussian Process) and comparison against VBLL](sngp_pytorch.ipynb)
* Implementation of SpectralNormalization wrapper
* Implementation of RandomFeatureGaussianProcess
* Implementation of DeepResNet SNGP
* Test of replacing classification head with [VBLL layers](https://github.com/VectorInstitute/vbll/)
7. [Bias of predictive mean calculated with averages of Dropout layers](dropout_bnn_mean.ipynb)
* Compare predictive means calculated by averaging samples from a BNN vs output calculated for Dropout averages.
6. [Comparison of REINFORCE vs Gumbel-Softmax vs MDNF gradients and convergence for a simplified objective](reinforce_vs_gumbelsoftmax_gradients.ipynb)
* Optimization using REINFORCE vs reparametrization gradients (with GradientTape)
* Gumbel-Softmax relaxation for discrete variables - an illustration of a bias
* Mixture of Discrete Normalizing Flows relaxation for discrete variables
5. [Illustration of how entropy of the relaxed categorical distribution can be estimated and utilized for VI](entropy_of_relaxed_categorical_distribution.ipynb)
* Comparison (and discussion of gradients) of three estimates of the entropy/KL-term in ELBO
4. [Variational Autoencoder using Relaxed Categorical distribution](vae_relaxed_categorical.ipynb)
* Sampling from Gumbel softmax with and without straight-through
* Implementation of different approaches to estimation of KL divergence
* Training with Mnist data
* Reconstruction of digits and unconditional sampling latent codes
3. [A demonstration of Discrete Flows: Invertible Generative Models of Discrete Data](discrete_flows.ipynb)
* Arithmetic on one-hot encoded vectors
* Trainig simple discrete transformation
* MLE-training of an autoregressive flow with masked autoencoder to match a target distribution.
2. [Probabilistc Matrix Factorization model with mean-field variational inference](probabilisitc_matrix_factorization_vi.ipynb).
* Probabilistc Matrix Factorization implementation
* estimating ELBO using MC
* training using pyTorch automatic differentiation
* simple evaluation of RMSE on test subset
1. [Framing multi-output Bayesian optimization with GPyOpt](multi-task_bayesian_optimization_demo.ipynb)
* fitting individual GPs
* fitting multi-task GPs using coregionalization
* BO optimization of single function
* BO optimization of 2-task problem
* An implementation of a custom acquisition function
* Extensive visualization of the optimization process