{"id":29624920,"url":"https://github.com/vincentstimper/normalizing-flows","last_synced_at":"2025-07-21T06:03:45.496Z","repository":{"id":37841429,"uuid":"236765661","full_name":"VincentStimper/normalizing-flows","owner":"VincentStimper","description":"PyTorch implementation of normalizing flow models","archived":false,"fork":false,"pushed_at":"2024-08-25T09:44:06.000Z","size":7804,"stargazers_count":859,"open_issues_count":12,"forks_count":128,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-07-19T06:28:17.341Z","etag":null,"topics":["density-estimation","glow","invertible-neural-networks","neural-spline-flow","normalizing-flow","pytorch","real-nvp","residual-flow","variational-autoencoder","variational-inference"],"latest_commit_sha":null,"homepage":"https://joss.theoj.org/papers/10.21105/joss.05361","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VincentStimper.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["VincentStimper"]}},"created_at":"2020-01-28T15:15:59.000Z","updated_at":"2025-07-18T09:35:51.000Z","dependencies_parsed_at":"2023-02-13T00:30:26.974Z","dependency_job_id":"d97cd5a8-ce15-4002-8077-f41677977096","html_url":"https://github.com/VincentStimper/normalizing-flows","commit_stats":{"total_commits":668,"total_committers":11,"mean_commits":60.72727272727273,"dds":"0.16017964071856283","last_synced_commit":"0466e7f530b5091e60eb35f6ce00ce004fd8845f"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/VincentStimper/normalizing-flows","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VincentStimper%2Fnormalizing-flows","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VincentStimper%2Fnormalizing-flows/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VincentStimper%2Fnormalizing-flows/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VincentStimper%2Fnormalizing-flows/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VincentStimper","download_url":"https://codeload.github.com/VincentStimper/normalizing-flows/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VincentStimper%2Fnormalizing-flows/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266248501,"owners_count":23899056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["density-estimation","glow","invertible-neural-networks","neural-spline-flow","normalizing-flow","pytorch","real-nvp","residual-flow","variational-autoencoder","variational-inference"],"created_at":"2025-07-21T06:03:03.487Z","updated_at":"2025-07-21T06:03:45.491Z","avatar_url":"https://github.com/VincentStimper.png","language":"Python","funding_links":["https://github.com/sponsors/VincentStimper"],"categories":[],"sub_categories":[],"readme":"# `normflows`: A PyTorch Package for Normalizing Flows\n\n[![documentation](https://github.com/VincentStimper/normalizing-flows/actions/workflows/mkdocs.yaml/badge.svg)](https://vincentstimper.github.io/normalizing-flows/)\n![unit-tests](https://github.com/VincentStimper/normalizing-flows/actions/workflows/pytest.yaml/badge.svg)\n![code coverage](https://raw.githubusercontent.com/VincentStimper/normalizing-flows/coverage-badge/coverage.svg?raw=true)\n[![License: MIT](https://img.shields.io/badge/Licence-MIT-b31b1b.svg)](https://opensource.org/licenses/MIT)\n[![DOI](https://joss.theoj.org/papers/10.21105/joss.05361/status.svg)](https://doi.org/10.21105/joss.05361)\n[![PyPI](https://img.shields.io/badge/PyPI-1.7.3-blue.svg)](https://pypi.org/project/normflows/)\n[![Downloads](https://static.pepy.tech/personalized-badge/normflows?period=total\u0026units=international_system\u0026left_color=grey\u0026right_color=orange\u0026left_text=Downloads)](https://pepy.tech/project/normflows)\n\n\n`normflows` is a PyTorch implementation of discrete normalizing flows. Many popular flow architectures are implemented,\nsee the [list below](#implemented-flows). The package can be easily [installed via pip](#installation).\nThe basic usage is described [here](#usage), and a [full documentation](https://vincentstimper.github.io/normalizing-flows/) \nis available as well. A more detailed description of this package is given in our\n[accompanying paper](https://joss.theoj.org/papers/10.21105/joss.05361).\n\nSeveral sample use cases are provided in the \n[`examples` folder](https://github.com/VincentStimper/normalizing-flows/blob/master/examples), \nincluding [Glow](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/glow.ipynb),\na [VAE](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/vae.py), and\na [Residual Flow](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/residual.ipynb).\nMoreover, two simple applications are highlighed in the [examples section](#examples). You can run them \nyourself in Google Colab using the links below to get a feeling for `normflows`.\n\n| Link                                                                                                                                                                                                                                                  | Description                                                             |\n|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|\n| \u003ca href=\"https://colab.research.google.com/github/VincentStimper/normalizing-flows/blob/master/examples/real_nvp_colab.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e          | Real NVP applied to a 2D bimodal target distribution                    |\n| \u003ca href=\"https://colab.research.google.com/github/VincentStimper/normalizing-flows/blob/master/examples/paper_example_nsf_colab.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e | Modeling a distribution on a cylinder surface with a neural spline flow |\n| \u003ca href=\"https://colab.research.google.com/github/VincentStimper/normalizing-flows/blob/master/examples/glow_colab.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e              | Modeling and generating CIFAR-10 images with Glow                       |\n\n\n## Implemented Flows\n\n| Architecture | Reference                                                                                                                 |\n|--------------|---------------------------------------------------------------------------------------------------------------------------|\n| Planar Flow  | [Rezende \u0026 Mohamed, 2015](http://proceedings.mlr.press/v37/rezende15.html)                                                |\n| Radial Flow  | [Rezende \u0026 Mohamed, 2015](http://proceedings.mlr.press/v37/rezende15.html)                                                |\n| NICE         | [Dinh et al., 2014](https://arxiv.org/abs/1410.8516)                                                                      |\n| Real NVP     | [Dinh et al., 2017](https://openreview.net/forum?id=HkpbnH9lx)                                                            |\n| Glow         | [Kingma et al., 2018](https://proceedings.neurips.cc/paper/2018/hash/d139db6a236200b21cc7f752979132d0-Abstract.html)                                                                   |\n| Masked Autoregressive Flow | [Papamakarios et al., 2017](https://proceedings.neurips.cc/paper/2017/hash/6c1da886822c67822bcf3679d04369fa-Abstract.html) |\n| Neural Spline Flow | [Durkan et al., 2019](https://proceedings.neurips.cc/paper/2019/hash/7ac71d433f282034e088473244df8c02-Abstract.html)                                                                    |\n| Circular Neural Spline Flow | [Rezende et al., 2020](http://proceedings.mlr.press/v119/rezende20a.html)                                                 |\n| Residual Flow | [Chen et al., 2019](https://proceedings.neurips.cc/paper/2019/hash/5d0d5594d24f0f955548f0fc0ff83d10-Abstract.html)                                                                     |\n| Stochastic Normalizing Flow | [Wu et al., 2020](https://proceedings.neurips.cc/paper/2020/hash/41d80bfc327ef980528426fc810a6d7a-Abstract.html)                                                                       |\n\nNote that Neural Spline Flows with circular and non-circular coordinates\nare supported as well.\n\n## Installation\n\nThe latest version of the package can be installed via pip\n\n```\npip install normflows\n```\n\nAt least Python 3.7 is required. If you want to use a GPU, make sure that\nPyTorch is set up correctly by following the instructions at the\n[PyTorch website](https://pytorch.org/get-started/locally/).\n\nTo run the example notebooks clone the repository first\n\n```\ngit clone https://github.com/VincentStimper/normalizing-flows.git\n```\n\nand then install the dependencies.\n\n```\npip install -r requirements_examples.txt\n```\n\n## Usage\n\nA normalizing flow consists of a base distribution, defined in \n[`nf.distributions.base`](https://github.com/VincentStimper/normalizing-flows/blob/master/normflows/distributions/base.py),\nand a list of flows, given in\n[`nf.flows`](https://github.com/VincentStimper/normalizing-flows/blob/master/normflows/flows).\nLet's assume our target is a 2D distribution. We pick a diagonal Gaussian\nbase distribution, which is the most popular choice. Our flow shall be a\n[Real NVP model](https://openreview.net/forum?id=HkpbnH9lx) and, therefore, we need\nto define a neural network for computing the parameters of the affine coupling\nmap. One dimension is used to compute the scale and shift parameter for the\nother dimension. After each coupling layer we swap their roles.\n\n```python\nimport normflows as nf\n\n# Define 2D Gaussian base distribution\nbase = nf.distributions.base.DiagGaussian(2)\n\n# Define list of flows\nnum_layers = 32\nflows = []\nfor i in range(num_layers):\n    # Neural network with two hidden layers having 64 units each\n    # Last layer is initialized by zeros making training more stable\n    param_map = nf.nets.MLP([1, 64, 64, 2], init_zeros=True)\n    # Add flow layer\n    flows.append(nf.flows.AffineCouplingBlock(param_map))\n    # Swap dimensions\n    flows.append(nf.flows.Permute(2, mode='swap'))\n```\n\nOnce they are set up, we can define a\n[`nf.NormalizingFlow`](https://github.com/VincentStimper/normalizing-flows/blob/master/normflows/core.py#L9)\nmodel. If the target density is available, it can be added to the model\nto be used during training. Sample target distributions are given in\n[`nf.distributions.target`](https://github.com/VincentStimper/normalizing-flows/blob/master/normflows/distributions/target.py).\n\n```python\n# If the target density is not given\nmodel = nf.NormalizingFlow(base, flows)\n\n# If the target density is given\ntarget = nf.distributions.target.TwoMoons()\nmodel = nf.NormalizingFlow(base, flows, target)\n```\n\nThe loss can be computed with the methods of the model and minimized.\n\n```python\n# When doing maximum likelihood learning, i.e. minimizing the forward KLD\n# with no target distribution given\nloss = model.forward_kld(x)\n\n# When minimizing the reverse KLD based on the given target distribution\nloss = model.reverse_kld(num_samples=512)\n\n# Optimization as usual\nloss.backward()\noptimizer.step()\n```\n\n## Examples\n\nWe provide several illustrative examples of how to use the package in the\n[`examples`](https://github.com/VincentStimper/normalizing-flows/blob/master/examples)\ndirectory. Among them are implementations of \n[Glow](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/glow.ipynb),\na [VAE](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/vae.py), and\na [Residual Flow](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/residual.ipynb). \nMore advanced experiments can be done with the scripts listed in the\n[repository about resampled base distributions](https://github.com/VincentStimper/resampled-base-flows),\nsee its [`experiments`](https://github.com/VincentStimper/resampled-base-flows/tree/master/experiments)\nfolder.\n\nBelow, we consider two simple 2D examples.\n\n### Real NVP applied to a 2D bimodal target distribution\n\n\u003ca href=\"https://colab.research.google.com/github/VincentStimper/normalizing-flows/blob/master/examples/real_nvp_colab.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n\nIn [this notebook](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/real_nvp_colab.ipynb), \nwhich can directly be opened in \n[Colab](https://colab.research.google.com/github/VincentStimper/normalizing-flows/blob/master/examples/real_nvp_colab.ipynb),\nwe consider a 2D distribution with two half-moon-shaped modes as a target. We approximate it with a Real NVP model\nand obtain the following results.\n\n![2D target distribution and Real NVP model](https://raw.githubusercontent.com/VincentStimper/normalizing-flows/master/figures/real_nvp.png)\n\nNote that there might be a density filament connecting the two modes, which is due to an architectural limitation \nof normalizing flows, especially prominent in Real NVP. You can find out more about it in \n[this paper](https://proceedings.mlr.press/v151/stimper22a).\n\n### Modeling a distribution on a cylinder surface with a neural spline flow\n\n\u003ca href=\"https://colab.research.google.com/github/VincentStimper/normalizing-flows/blob/master/examples/paper_example_nsf_colab.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n\nIn [another example](https://github.com/VincentStimper/normalizing-flows/blob/master/examples/paper_example_nsf_colab.ipynb),\nwhich is available in [Colab](https://colab.research.google.com/github/VincentStimper/normalizing-flows/blob/master/examples/paper_example_nsf_colab.ipynb)\nas well, we apply a Neural Spline Flow model to a distribution defined on a cylinder. The resulting density is visualized below.\n\n![Neural Spline Flow applied to target distribution on a cylinder](https://raw.githubusercontent.com/VincentStimper/normalizing-flows/master/figures/nsf_cylinder_3d.png)\n\nThis example is considered in the [paper](https://joss.theoj.org/papers/10.21105/joss.05361) accompanying this repository.\n\n## Support\n\nIf you have problems, please read the [package documentation](https://vincentstimper.github.io/normalizing-flows/)\nand check out the [examples section](#examples) above. You are also welcome to \n[create issues on GitHub](https://github.com/VincentStimper/normalizing-flows/issues) to get help. Note that it is\nworthwhile browsing the existing [open](https://github.com/VincentStimper/normalizing-flows/issues?q=is%3Aopen+is%3Aissue) \nand [closed](https://github.com/VincentStimper/normalizing-flows/issues?q=is%3Aissue+is%3Aclosed) issues, which might\naddress the problem you are facing.\n\n## Contributing\n\nIf you find a bug or have a feature request, please \n[file an issue on GitHub](https://github.com/VincentStimper/normalizing-flows/issues).\n\nYou are welcome to contribute to the package by fixing the bug or adding the feature yourself. If you want to \ncontribute, please add tests for the code you added or modified and ensure it passes successfully by running `pytest`.\nThis can be done by simply executing\n```\npytest\n```\nwithin your local version of the repository. Make sure you code is well documented, and we also encourage contributions\nto the existing documentation. Once you finished coding and testing, please \n[create a pull request on GitHub](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).\n\n## Used by\n\nThe package has been used in several research papers. Some of them are listed below.\n\n\u003e Andrew Campbell, Wenlong Chen, Vincent Stimper, José Miguel Hernández-Lobato, and Yichuan Zhang. \n\u003e [A gradient based strategy for Hamiltonian Monte Carlo hyperparameter optimization](https://proceedings.mlr.press/v139/campbell21a.html). \n\u003e In Proceedings of the 38th International Conference on Machine Learning, pp. 1238–1248. PMLR, 2021.\n\u003e \n\u003e [Code available on GitHub.](https://github.com/VincentStimper/hmc-hyperparameter-tuning)\n\n\u003e Vincent Stimper, Bernhard Schölkopf, and José Miguel Hernández-Lobato. \n\u003e [Resampling Base Distributions of Normalizing Flows](https://proceedings.mlr.press/v151/stimper22a). \n\u003e In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151, pp. 4915–4936, 2022.\n\u003e \n\u003e [Code available on GitHub.](https://github.com/VincentStimper/resampled-base-flows)\n\n\u003e Laurence I. Midgley, Vincent Stimper, Gregor N. C. Simm, Bernhard Schölkopf, and José Miguel Hernández-Lobato. \n\u003e [Flow Annealed Importance Sampling Bootstrap](https://arxiv.org/abs/2208.01893). \n\u003e The Eleventh International Conference on Learning Representations, 2023.\n\u003e \n\u003e [Code available on GitHub.](https://github.com/lollcat/fab-torch)\n\n\u003e Arnau Quera-Bofarull, Joel Dyer, Anisoara Calinescu, J. Doyne Farmer, and Michael Wooldridge.\n\u003e [BlackBIRDS: Black-Box Inference foR Differentiable Simulators](https://joss.theoj.org/papers/10.21105/joss.05776).\n\u003e Journal of Open Source Software, 8(89), 5776, 2023.\n\u003e \n\u003e [Code available on GitHub.](https://github.com/arnauqb/blackbirds)\n\n\u003e Utkarsh Singhal, Carlos Esteves, Ameesh Makadia, and Stella X. Yu.\n\u003e [Learning to Transform for Generalizable Instance-wise Invariance](https://openaccess.thecvf.com/content/ICCV2023/html/Singhal_Learning_to_Transform_for_Generalizable_Instance-wise_Invariance_ICCV_2023_paper.html).\n\u003e Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6211-6221, 2023.\n\u003e \n\u003e [Code available on GitHub.](https://github.com/sutkarsh/flow_inv)\n\n\u003e Ba-Hien Tran, Giulio Franzese, Pietro Michiardi, and Maurizio Filippone.\n\u003e [One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Models](https://proceedings.neurips.cc/paper_files/paper/2023/hash/1516a7f7507d5550db5c7f29e995ec8c-Abstract-Conference.html).\n\u003e  Advances in Neural Information Processing Systems 36, pp. 6545–6567, 2023.\n\u003e \n\u003e [Code available on GitHub.](https://github.com/tranbahien/data-mollification)\n\nMoreover, the [`boltzgen`](https://github.com/VincentStimper/boltzmann-generators) package\nhas been build upon `normflows`.\n\n## Citation\n\nIf you use `normflows`, please cite the \n[corresponding paper](https://joss.theoj.org/papers/10.21105/joss.05361) as follows.\n\n\u003e Stimper et al., (2023). normflows: A PyTorch Package for Normalizing Flows. \n\u003e Journal of Open Source Software, 8(86), 5361, https://doi.org/10.21105/joss.05361\n\n**Bibtex**\n\n```\n@article{Stimper2023, \n  author = {Vincent Stimper and David Liu and Andrew Campbell and Vincent Berenz and Lukas Ryll and Bernhard Schölkopf and José Miguel Hernández-Lobato}, \n  title = {normflows: A PyTorch Package for Normalizing Flows}, \n  journal = {Journal of Open Source Software}, \n  volume = {8},\n  number = {86}, \n  pages = {5361}, \n  publisher = {The Open Journal}, \n  doi = {10.21105/joss.05361}, \n  url = {https://doi.org/10.21105/joss.05361}, \n  year = {2023}\n} \n```\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvincentstimper%2Fnormalizing-flows","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvincentstimper%2Fnormalizing-flows","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvincentstimper%2Fnormalizing-flows/lists"}