{"id":20685212,"url":"https://github.com/marpogaus/bernstein_flow","last_synced_at":"2025-04-22T13:39:53.142Z","repository":{"id":47900264,"uuid":"281648694","full_name":"MArpogaus/bernstein_flow","owner":"MArpogaus","description":"A normalizing flow using Bernstein polynomials for conditional density estimation.","archived":false,"fork":false,"pushed_at":"2024-10-18T15:46:12.000Z","size":16236,"stargazers_count":19,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-29T15:34:43.309Z","etag":null,"topics":["bernstein-polynomials","machine-learning","probabilistic-modeling","regression","tensorflow-probability","tensorflow2"],"latest_commit_sha":null,"homepage":"https://marpogaus.github.io/bernstein_flow/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MArpogaus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-22T10:36:26.000Z","updated_at":"2025-02-04T14:13:05.000Z","dependencies_parsed_at":"2024-06-13T20:26:50.230Z","dependency_job_id":"13318e6c-1ea0-4c58-b02f-a43a37e2daf7","html_url":"https://github.com/MArpogaus/bernstein_flow","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MArpogaus%2Fbernstein_flow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MArpogaus%2Fbernstein_flow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MArpogaus%2Fbernstein_flow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MArpogaus%2Fbernstein_flow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MArpogaus","download_url":"https://codeload.github.com/MArpogaus/bernstein_flow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250250094,"owners_count":21399584,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bernstein-polynomials","machine-learning","probabilistic-modeling","regression","tensorflow-probability","tensorflow2"],"created_at":"2024-11-16T22:26:23.860Z","updated_at":"2025-04-22T13:39:53.119Z","avatar_url":"https://github.com/MArpogaus.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Contributors][contributors-shield]][contributors-url]\n[![Forks][forks-shield]][forks-url]\n[![Stargazers][stars-shield]][stars-url]\n[![Issues][issues-shield]][issues-url]\n[![Apache License 2.0][license-shield]][license-url]\n[![LinkedIn][linkedin-shield]][linkedin-url]\n\n# Bernstein-Polynomials as TensorFlow Probability Bijector\n\nThis Repository contains an implementation of a normalizing flow for conditional density estimation using Bernstein polynomials, as proposed in:\n\n\u003e Sick Beate, Hothorn Torsten and Dürr Oliver, *Deep transformation models: Tackling complex regression problems with neural network based transformation models*, 2020. [online](http://arxiv.org/abs/2004.00464)\n\nThe [`tfp.Bijector`][bijector] interface is used for the implementation to benefit from the powerful [TensorFlow Probability][tensorflow-probability] framework.\n\n\u003c!-- MarkdownTOC levels=2 --\u003e\n\n- [The Need for Flexible Distributions](#the-need-for-flexible-distributions)\n- [Getting Started](#getting-started)\n- [Usage](#usage)\n- [Examples](#examples)\n- [Contributing](#contributing)\n- [License](#license)\n\n\u003c!-- /MarkdownTOC --\u003e\n\n\n## The Need for Flexible Distributions\n\nTraditional regression models assume normality and homoscedasticity of the data, i.e. the residuals for each input value are expected to be normally distributed with constant variance.\nHowever, the shape of the data distribution in many real use cases is much more complex.\n\nThe following example of a classical data set containing the waiting time between eruptions of the [Old Faithful Geyser](https://en.wikipedia.org/wiki/Old_Faithful) in [Yellowstone National Park](https://en.wikipedia.org/wiki/Yellowstone_National_Park) is used as an example.\n\n| Gaussian                        | Normalizing Flow              |\n|:--------------------------------|:------------------------------|\n| ![gauss](./ipynb/gfx/gauss.png) | ![flow](./ipynb/gfx/flow.png) |\n\nAs shown in the left figure, the normality assumption is clearly violated by the bimodal nature of the data.\nHowever, the proposed transformation model has the flexibility to adapt to this complexity.\n\n## Getting Started\n\nTo start using my code follow these simple steps.\n\n### Installation\n\nPull and install it directly from git using pip:\n\n```bash\npip install git+https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector.git\n```\n\nOr clone this repository and install it from there:\n\n```bash\ngit clone https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector.git ./bernstein_flow\ncd bernstein_flow\npip install -e .\n```\n\n### Prerequisites\n\nPip should take care of installing the required dependencies on its own.\nFor completeness, these are the packages used in the implementation:\n\n * [`matplotlib`][matplotlib]\n * [`numpy`][numpy]\n * [`scipy`][scipy]\n * [`tensorflow`][tensorflow]\n * [`tensorflow_probability`][tensorflow-probability]\n\n## Usage\n\n### Package Structure \n\nThis python package consists of four main components:\n\n * `berstein_flow.bijectors.BernsteinBijector`: The implementation of Bernstein polynomials using the `tfb.Bijector` interface for\n    transformations of `tfd.Distribution` samples.\n * `berstein_flow.distributions.BernsteinFlow`:  The implementation of a `tfd.TransformedDistribution` using the Bernstein\n    polynomials as the bijector.\n * `berstein_flow.losses.BernsteinFlowLoss`: The implementation of a `tfk.losses.Loss` function to calculate the negative logarithmic likelihood using the `BernstinFlow` distribution.\n * `berstein_flow.util.visualization`: Contains of some convenient helper functions for visualization.\n\n### Using the Model as a `tfpl.DistributionLambda`\n\nA [`tfd.TransformedDistribution`][transformed-distribution] using the [BernsteinBijector][bernstein-bijector] is provided in the module `bernstein_flow.distributions.BernsteinFlow`:\n\n```python\nfrom bernstein_flow.distributions import BernsteinFlow\n```\n\nUse it like any other distribution, i.e. as a [`tfpl.DistributionLambda`][distribution-lambda].\n\nThe two example plots shown above have been generated using the following two models.\n\n### Gaussian Model\n\n```python\ngauss_model = tf.keras.Sequential()\ngauss_model.add(InputLayer(input_shape = (1)))\n#Here could come a gigantus network\ngauss_model.add(Dense(2)) # mean and the std of the Gaussian\ngauss_model.add(tfp.layers.DistributionLambda(\n    lambda pv:\n        tfd.Normal(loc=pv[:,0], scale=1e-3 + tf.math.softplus(0.05 * pv[:,1]))))\n```\n\n### Normalizing FLow\n\n```python\nflow_model = tf.keras.Sequential()\nflow_model.add(InputLayer(input_shape = (1)))\n#Here could come a gigantus network\nflow_model.add(Dense(4 + 5)) # Bernstein coefficients and 2 times scale and shift\nflow_model.add(tfp.layers.DistributionLambda(BernsteinFlow))\n```\n\n## Examples\n\nYou can find two examples in the `ipynb` directory:\n\n * `TheoreticalBackground.ipynb`: Some explanation of the theoretical fundamentals\n * `Gaussian_vs_Transformation_Model.ipynb`: Bimodal data example shown in the figures above.\n\n## Contributing\n\nIf you have any technical issues or suggestion regarding my implementation, please feel free to either [contact me](mailto:marcel.arpogaus@gmail.com), [open an issue][open-an-issue] or send me a Pull Request:\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\nAny contributions are **greatly appreciated**.\n\n## License\n\nDistributed under the [Apache License 2.0](LICENSE)\n\n[contributors-shield]: https://img.shields.io/github/contributors/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector.svg?style=flat-square\n[contributors-url]: https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector.svg?style=flat-square\n[forks-url]: https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector/network/members\n[stars-shield]: https://img.shields.io/github/stars/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector.svg?style=flat-square\n[stars-url]: https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector/stargazers\n[issues-shield]: https://img.shields.io/github/issues/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector.svg?style=flat-square\n[issues-url]: https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector/issues\n[license-shield]: https://img.shields.io/github/license/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector.svg?style=flat-square\n[license-url]: https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector/blob/master/LICENSE\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=flat-square\u0026logo=linkedin\u0026colorB=555\n[linkedin-url]: https://linkedin.com/in/MArpogaus\n[bijector]: https://www.tensorflow.org/probability/api_docs/python/tfp/bijectors/Bijector\n[tensorflow-probability]: https://www.tensorflow.org/probability\n[matplotlib]: https://matplotlib.org/\n[numpy]: https://numpy.org/\n[scipy]: https://scipy.org/\n[tensorflow]: https://www.tensorflow.org/\n[transformed-distribution]: https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/TransformedDistribution\n[bernstein-bijector]: https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector/blob/master/src/bernstein_flow/bijectors/bernstein_bijector.py\n[distribution-lambda]: https://www.tensorflow.org/probability/api_docs/python/tfp/layers/DistributionLambda\n[open-an-issue]: https://github.com/MArpogaus/TensorFlow-Probability-Bernstein-Polynomial-Bijector/issues/new\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarpogaus%2Fbernstein_flow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarpogaus%2Fbernstein_flow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarpogaus%2Fbernstein_flow/lists"}