{"id":16510776,"url":"https://github.com/janosh/tf-mnf","last_synced_at":"2025-10-28T03:31:57.368Z","repository":{"id":41881500,"uuid":"238915745","full_name":"janosh/tf-mnf","owner":"janosh","description":"Multiplicative normalizing flow: variational Bayesian neural networks with increased posterior flexibility thanks to normalizing flows","archived":false,"fork":false,"pushed_at":"2025-01-06T16:33:28.000Z","size":1328,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-02-01T11:11:06.421Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/janosh.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-02-07T12:25:37.000Z","updated_at":"2024-08-30T09:55:33.000Z","dependencies_parsed_at":"2024-01-01T17:42:17.937Z","dependency_job_id":null,"html_url":"https://github.com/janosh/tf-mnf","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janosh%2Ftf-mnf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janosh%2Ftf-mnf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janosh%2Ftf-mnf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/janosh%2Ftf-mnf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/janosh","download_url":"https://codeload.github.com/janosh/tf-mnf/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238590596,"owners_count":19497350,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T15:57:15.326Z","updated_at":"2025-10-28T03:31:56.237Z","avatar_url":"https://github.com/janosh.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TF-MNF\n\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/janosh/tf-mnf/master.svg)](https://results.pre-commit.ci/latest/github/janosh/tf-mnf/master)\n[![License](https://img.shields.io/github/license/janosh/tf-mnf?label=License)](/license)\n![GitHub Repo Size](https://img.shields.io/github/repo-size/janosh/tf-mnf?label=Repo+Size)\n![GitHub last commit](https://img.shields.io/github/last-commit/janosh/tf-mnf?label=Last+Commit)\n\n\u003cimg src=\"assets/normalizing-flow.svg\" alt=\"Normalizing Flow\" align=\"right\" height=\"150\"\u003e\n\nTensorFlow 2.0 implementation of Multiplicative Normalizing Flows (MNF). MNF is a Bayesian neural network variant introduced in [[1]](#tf-mnf) with auxiliary random variables and a factorial Gaussian posterior with means `mu_i` conditioned on scaling factors `z_i` modelled by normalizing flows.\n\nImplements the planar [[3]](#vi-nf), RNVP [[4]](#rnvp) and MAF [[5]](#maf) flow. MAF uses the MADE autoregressive network architecture introduced in [[2]](#made)\n\n## References\n\n1. \u003ca id=\"mnf-vbnn\"\u003e\u003c/a\u003e **_Multiplicative Normalizing Flows for Variational Bayesian Neural Networks_** | Christos Louizos, Max Welling (Mar 2017) | [1703.01961](https://arxiv.org/abs/1703.01961)\n\n2. \u003ca id=\"made\"\u003e\u003c/a\u003e **_MADE: Masked Autoencoder for Distribution Estimation_** | Mathieu Germain, Karol Gregor, Iain Murray, Hugo Larochelle (Jun 2015) | [1502.03509](https://arxiv.org/abs/1502.03509)\n\n3. \u003ca id=\"vi-nf\"\u003e\u003c/a\u003e **_Variational Inference with Normalizing Flows_** | Danilo Rezende, Shakir Mohamed (May 2015) | [1505.05770](https://arxiv.org/abs/1505.05770)\n\n4. \u003ca id=\"rnvp\"\u003e\u003c/a\u003e **_Density estimation using Real NVP_** | Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio (May 2016) | [1605.08803](https://arxiv.org/abs/1605.08803)\n\n5. \u003ca id=\"maf\"\u003e\u003c/a\u003e **_Masked Autoregressive Flow for Density Estimation_** | George Papamakarios, Theo Pavlakou, Iain Murray (Jun 2018) | [1705.07057](https://arxiv.org/abs/1705.07057)\n\n6. \u003ca id=\"bay-hyp\"\u003e\u003c/a\u003e **_Bayesian Hypernetworks_** | David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, Aaron Courville (Oct 2017) | [1710.04759](https://arxiv.org/abs/1710.04759)\n\n## MNF-LeNet vs regular LeNet5 on rotating MNIST\n\n|                       MNF-LeNet                        |                     Regular LeNet                      |\n| :----------------------------------------------------: | :----------------------------------------------------: |\n| \u003cimg src=\"assets/rot-9-mnf-lenet.svg\" alt=\"MNF-LeNet\"\u003e | \u003cimg src=\"assets/rot-9-lenet.svg\" alt=\"Regular LeNet\"\u003e |\n\nA good test for Bayesian vision models is to take an MNIST 9 **and** rotate it in steps of 20 degree until we're back to a 6. This evaluates how the model and its uncertainty estimates perform as we slowly exit and then re-enter the training manifold.\n\nThe plots show how MNF applied to a [LeNet model](http://yann.lecun.com/exdb/lenet) compares to an unmodified LeNet. The code for these plots is found in [`notebooks/lenet_mnist.py`](/tf_mnf/notebooks/lenet_mnist.py).\n\nIn the top left, both models start out almost certain they're being shown a 9. A rotation of 20 degree already causes the MNF-LeNet to allocate small probabilities to other digits. The middle row is the most interesting since we're farthest away from the training manifold. MNF-LeNet distributes its softmax more evenly whereas the regular LeNet tends to focus its prediction on a single digit.\n\nWhile the regular LeNet only outputs point estimates, the Bayesian MNF-LeNet returns distributions. An interesting thing to note is that the violins in the top and middle rows exhibit some non-Gaussian structure. In particular, some seem to be multimodal, suggesting that there might be multiple clusters of dissimilar network parametrizations that have slightly different opinions on how likely that digit is.\n\n## Conda Environment\n\nThe environment file `env.yml` was generated with `conda env export --no-builds \u003e env.yml`. To recreate the environment from this file run `conda env create -f env.yml`.\n\nThe environment `tf-mnf` was originally created by running the command:\n\n```sh\nconda create -n tf-mnf python=3.6 \\\n  \u0026\u0026 conda activate tf-mnf \\\n  \u0026\u0026 pip install tensorflow tensorflow-probability jupyter tqdm seaborn\n```\n\nTo delete the environment run `conda env remove -n tf-mnf`.\n\nTo update all packages and reflect changes in this file use\n\n```sh\nconda update --all \\\n  \u0026\u0026 pip list --outdated --format=freeze | grep -v '^\\-e' | cut -d = -f 1  | xargs -n1 pip install -U \\\n  \u0026\u0026 conda env export --no-builds \u003e env.yml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjanosh%2Ftf-mnf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjanosh%2Ftf-mnf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjanosh%2Ftf-mnf/lists"}