{"id":13487071,"url":"https://github.com/ctallec/pyvarinf","last_synced_at":"2026-04-22T18:51:03.618Z","repository":{"id":57458285,"uuid":"123596343","full_name":"ctallec/pyvarinf","owner":"ctallec","description":"Python package facilitating the use of Bayesian Deep Learning methods with Variational Inference for PyTorch","archived":false,"fork":false,"pushed_at":"2019-10-12T14:55:49.000Z","size":49,"stargazers_count":361,"open_issues_count":4,"forks_count":50,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-09-12T02:09:34.843Z","etag":null,"topics":["bayesian-inference","deep-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ctallec.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-03-02T15:32:37.000Z","updated_at":"2025-08-19T11:52:36.000Z","dependencies_parsed_at":"2022-09-09T22:50:17.547Z","dependency_job_id":null,"html_url":"https://github.com/ctallec/pyvarinf","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/ctallec/pyvarinf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctallec%2Fpyvarinf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctallec%2Fpyvarinf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctallec%2Fpyvarinf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctallec%2Fpyvarinf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ctallec","download_url":"https://codeload.github.com/ctallec/pyvarinf/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ctallec%2Fpyvarinf/sbom","scorecard":{"id":311612,"data":{"date":"2025-08-11","repo":{"name":"github.com/ctallec/pyvarinf","commit":"50571c950ccb586b356ccdb10dcc8190ed22204b"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.2,"checks":[{"name":"Code-Review","score":1,"reason":"Found 4/26 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Info: FSF or OSI recognized license: MIT License: LICENSE.txt:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 9 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-17T23:16:00.520Z","repository_id":57458285,"created_at":"2025-08-17T23:16:00.521Z","updated_at":"2025-08-17T23:16:00.521Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32150396,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-22T17:06:48.269Z","status":"ssl_error","status_checked_at":"2026-04-22T17:06:19.037Z","response_time":58,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-inference","deep-learning"],"created_at":"2024-07-31T18:00:55.019Z","updated_at":"2026-04-22T18:51:03.315Z","avatar_url":"https://github.com/ctallec.png","language":"Python","funding_links":[],"categories":["The Data Science Toolbox","Probabilistic Methods","Pytorch \u0026 related libraries｜Pytorch \u0026 相关库","Pytorch \u0026 related libraries","Python"],"sub_categories":["Deep Learning Packages","NLP","Others","Probabilistic/Generative Libraries｜概率库和生成库:","Probabilistic/Generative Libraries:"],"readme":"# PyVarInf\nPyVarInf provides facilities to easily train your PyTorch neural network models using variational inference.\n\n# Bayesian Deep Learning with Variational Inference\n\n## Bayesian Deep Learning\nAssume we have a dataset D = {(x\u003csub\u003e1\u003c/sub\u003e, y\u003csub\u003e1\u003c/sub\u003e), ...,\n(x\u003csub\u003en\u003c/sub\u003e, y\u003csub\u003en\u003c/sub\u003e)} where the x's are the inputs and the y's the\noutputs. The problem is to predict the y's from the x's. Further assume that\np(D|θ) is the output of a neural network with *weights* θ. The *network loss*\nis defined as\n\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?L^n(\\theta)\u0026space;=\u0026space;-\\log\u0026space;p(D|\\theta)\u0026space;=\u0026space;-\\sum_i\u0026space;\\log\u0026space;p(y_i|x_i,\u0026space;\\theta)\" title=\"L^n(\\theta) = -\\log p(D|\\theta) = -\\sum_i \\log p(y_i|x_i, \\theta)\" /\u003e\n\nUsually, when training a neural network, we try to find the parameter θ* which minimizes L\u003csup\u003en\u003c/sup\u003e(θ).\n\nIn Bayesian Inference, the problem is instead to study the posterior\ndistribution of the weights given the data. Assume we have a prior α over\nℝ\u003csup\u003ed\u003c/sup\u003e. The posterior is\n\n\u003cimg\nsrc=\"https://latex.codecogs.com/gif.latex?p(\\theta|D)\u0026space;=\u0026space;\\frac{p(D|\\theta)\\alpha(\\theta)}{\\int_\\theta\u0026space;p(D|\\theta)\\alpha(\\theta)\u0026space;d\\theta}\"\ntitle=\"p(\\theta|D) = \\frac{p(D|\\theta)\\alpha(\\theta)}{\\int_\\theta\np(D|\\theta)\\alpha(\\theta) d\\theta}\" /\u003e\n\nThis can be used for model selection, or prediction with Bayesian Model Averaging.\n\n## Variational Inference\nIt is usually impossible to analytically compute the posterior distribution,\nespecially with models as complex as neural networks. Variational Inference adress\nthis problem by approximating the posterior p(θ|D) by a parametric distribution\nq(θ|φ) where φ is a parameter. The problem is then not to learn a parameter θ*\nbut a probability distribution q(θ|φ) minimizing \n\n\u003cimg src=\"https://latex.codecogs.com/gif.latex?F(\\phi)\u0026space;=\u0026space;\\mathbf{E}_{\\theta\u0026space;\\sim\u0026space;q(\\theta|\\phi)}[L^N(\\theta)]\u0026space;\u0026plus;\u0026space;KL(q(.|\\phi)\\|\\alpha)\" title=\"F(\\phi) = E_{\\theta \\sim q(\\theta|\\phi)}[L^N(\\theta)] + KL(q(.|\\phi)\\|\\alpha)\" /\u003e\n\nF is called the *variational free energy*.\n\nThis idea was originally introduced for deep learning by Hinton and Van Camp\n[5] as a way to use neural networks for Minimum Description Length [3].\nMDL aims at minimizing the number of bits used to encode the whole dataset.\nVariational inference introduces one of many data encoding schemes.\nIndeed, F can be interpreted as the total description length of the dataset D,\nwhen we first encode the model, then encode the part of the data not explained by\nthe model: \n* L\u003csup\u003eC\u003c/sup\u003e(φ) = KL(q(.|φ)||α) is the complexity loss. It measures (in\n  nats) the quantity of information contained in the model. It is indeed\n  possible to encode the model in L\u003csup\u003eC\u003c/sup\u003e(φ) nats, with the *bits-back*\n  code [4].\n* L\u003csup\u003eE\u003c/sup\u003e(φ) = __E__\u003csub\u003eθ ~ q(θ|φ)\u003c/sub\u003e[L\u003csup\u003en\u003c/sup\u003e(θ)] is the error\n  loss. It measures the necessary quantity of information for encoding the data\n  D with the model. This code length can be achieved with a Shannon-Huffman\n  code for instance.\n\nTherefore F(φ) = L\u003csup\u003eC\u003c/sup\u003e(φ) + L\u003csup\u003eE\u003c/sup\u003e(φ) can be rephrased as an\nMDL loss function which measures the total encoding length of the data.\n\n## Practical Variational Optimisation\nIn practice, we define φ = (µ, σ) in ℝ\u003csup\u003ed\u003c/sup\u003e x ℝ\u003csup\u003ed\u003c/sup\u003e, and q(.|φ)\n= N(µ, Σ) the multivariate distribution where Σ =\ndiag(σ\u003csub\u003e1\u003c/sub\u003e\u003csup\u003e2\u003c/sup\u003e, ..., σ\u003csub\u003ed\u003c/sub\u003e\u003csup\u003e2\u003c/sup\u003e), and we want to\nfind the optimal µ* and σ*.\n\nWith this choice of a gaussian posterior, a Monte Carlo estimate of the\ngradient of F w.r.t. µ and σ can be obtained with backpropagation. This allows\nto use any gradient descent method used for non-variational optimisation [2]\n\n\n# Overview of PyVarInf\nThe core feature of PyVarInf is the `Variationalize` function. `Variationalize`\ntakes a model as input and outputs a variationalized version of the model with\ngaussian posterior.\n\n## Definition of a variational model\nTo define a variational model, first define a traditional PyTorch model, then\nuse the Variationalize function : \n```python\nimport pyvarinf\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\n\nclass Net(nn.Module):\n    def __init__(self):\n        super(Net, self).__init__()\n        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n        self.fc1 = nn.Linear(320, 50)\n        self.fc2 = nn.Linear(50, 10)\n        self.bn1 = nn.BatchNorm2d(10)\n        self.bn2 = nn.BatchNorm2d(20)\n\n    def forward(self, x):\n        x = self.bn1(F.relu(F.max_pool2d(self.conv1(x), 2)))\n        x = self.bn2(F.relu(F.max_pool2d(self.conv2(x), 2)))\n        x = x.view(-1, 320)\n        x = F.relu(self.fc1(x))\n        x = self.fc2(x)\n        return F.log_softmax(x)\n\nmodel = Net()\nvar_model = pyvarinf.Variationalize(model)\nvar_model.cuda()\n```\n\n## Optimisation of a variational model\nThen, the `var_model` can be trained that way : \n```python\noptimizer = optim.Adam(var_model.parameters(), lr=0.01)\n\ndef train(epoch):\n    var_model.train()\n    for batch_idx, (data, target) in enumerate(train_loader):\n        data, target = data.cuda(), target.cuda()\n        data, target = Variable(data), Variable(target)\n        optimizer.zero_grad()\n        output = var_model(data)\n        loss_error = F.nll_loss(output, target)\n\t# The model is only sent once, thus the division by\n\t# the number of datapoints used to train\n        loss_prior = var_model.prior_loss() / 60000\n        loss = loss_error + loss_prior\n        loss.backward()\n        optimizer.step()\n\nfor epoch in range(1, 500):\n    train(epoch)\n```\n\n## Available priors\n\nIn PyVarInf, we have implemented four families of priors : \n\n### Gaussian prior\nThe gaussian prior is N(0,Σ), with Σ the diagonal matrix diag(σ\u003csub\u003e1\u003c/sub\u003e\u003csup\u003e2\u003c/sup\u003e, ..., σ\u003csub\u003ed\u003c/sub\u003e\u003csup\u003e2\u003c/sup\u003e) defined such that 1/σ\u003csub\u003ei\u003c/sub\u003e is the square root of the number of parameters in the layer, following the standard initialisation of neural network weights. \nIt is the default prior, and do not have any parameter. It can be set with : \n```python\nvar_model.set_prior('gaussian')\n```\n### Conjugate priors\nThe conjugate prior is used if we assume that all the weights in a given layer should be distributed as a gaussian, but with unknown mean and variance. See [6] for more details. This prior can be set with \n```python\nvar_model.set_prior('conjugate', n_mc_samples, alpha_0, beta_0, mu_0, kappa_0)\n```\nThere are five parameters that have to bet set : \n- `n_mc_samples`, the number of samples used in the Monte Carlo estimation of the prior loss and its gradient.\n* `mu_0`, the prior sample mean\n* `kappa_0`, the number of samples used to estimate the prior sample mean\n* `alpha_0` and `beta_0`, such that variance was estimated from 2 alpha_0 observations with sample mean mu_0 and sum of squared deviations 2 beta_0\n             \n### Conjugate prior with known mean\nThe conjugate prior with known mean is similar to the conjugate prior. It is used if we assume that all the weights in a given layer should be distributed as a gaussian with a known mean but unknown variance. It is usefull in neural networks model when we assume that the weights in a layer should have mean 0. See [6] for more details. This prior can be set with :\n```python\nvar_model.set_prior('conjugate_known_mean', n_mc_samples, mean, alpha_0, beta_0)\n```\nFour parameters have to be set:\n* `n_mc_samples`, the number of samples used in the Monte Carlo estimation of the prior loss and its gradient.\n* `mean`, the known mean\n* `alpha_0` and `beta_0` defined as above\n\n\n\n### Mixture of two gaussian\nThe idea of using a mixture of two gaussians is defined in [1]. This prior can be set with: \n```python\nvar_model.set_prior('mixtgauss', n_mc_samples, sigma_1, sigma_2, pi)\n```\n* `n_mc_samples`, the number of samples used in the Monte Carlo estimation of the prior loss and its gradient.\n* `sigma_1` and `sigma_2` the std of the two gaussians\n* `pi` the probability of the first gaussian\n\n# Requirements\nThis module requires Python 3. You need to have PyTorch installed for PyVarInf to work (as PyTorch is not readily available on PyPi). To install PyTorch, follow the instructions described [here](http://pytorch.org/#pip-install-pytorch).\n\n# References\n* [1] Blundell, Charles, Cornebise, Julien, Kavukcuoglu, Koray, and Wierstra, Daan. Weight Uncertainty in Neural Networks. In *International Conference on Machine Learning*, pp. 1613–1622, 2015.\n* [2] Graves, Alex. Practical Variational Inference for Neural Networks. In *Neural Information Processing Systems*, 2011.\n* [3] Grünwald, Peter D. *The Minimum Description Length principle*. MIT press, 2007.\n* [4] Honkela, Antti and Valpola, Harri. Variational Learning and Bits-Back Coding: An Information-Theoretic View to Bayesian Learning. *IEEE transactions on Neural Networks*, 15(4), 2004.\n* [5] Hinton, Geoffrey E and Van Camp, Drew. Keeping Neural Networks Simple by Minimizing the Description Length of the Weights. In *Proceedings of the sixth annual conference on Computational learning theory*. ACM, 1993.\n* [6] Murphy, Kevin P. *Conjugate Bayesian analysis of the Gaussian distribution.*, 2007.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fctallec%2Fpyvarinf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fctallec%2Fpyvarinf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fctallec%2Fpyvarinf/lists"}