{"id":15066215,"url":"https://github.com/lightning-uq-box/lightning-uq-box","last_synced_at":"2025-05-16T12:10:50.704Z","repository":{"id":209019383,"uuid":"602910106","full_name":"lightning-uq-box/lightning-uq-box","owner":"lightning-uq-box","description":"Lightning-UQ-Box: Uncertainty Quantification for Neural Networks with PyTorch and Lightning","archived":false,"fork":false,"pushed_at":"2025-05-08T21:02:59.000Z","size":49237,"stargazers_count":177,"open_issues_count":23,"forks_count":18,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-05-08T22:20:14.321Z","etag":null,"topics":["bayesian-neural-networks","conformal-prediction","deep-learning","earth-observation","predictive-uncertainty","pytorch","pytorch-lightning","uncertainty-quantification"],"latest_commit_sha":null,"homepage":"https://lightning-uq-box.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lightning-uq-box.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-17T07:50:28.000Z","updated_at":"2025-05-08T21:02:56.000Z","dependencies_parsed_at":"2024-01-11T23:08:34.895Z","dependency_job_id":"f54fde8a-8ab4-4796-9c2d-aed16944a148","html_url":"https://github.com/lightning-uq-box/lightning-uq-box","commit_stats":{"total_commits":262,"total_committers":10,"mean_commits":26.2,"dds":0.5458015267175573,"last_synced_commit":"7a02f9a5c0e3edeb7d1aa47a9dd7ecb1e859192a"},"previous_names":["lightning-uq-box/lightning-uq-box"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightning-uq-box%2Flightning-uq-box","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightning-uq-box%2Flightning-uq-box/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightning-uq-box%2Flightning-uq-box/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightning-uq-box%2Flightning-uq-box/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lightning-uq-box","download_url":"https://codeload.github.com/lightning-uq-box/lightning-uq-box/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254011038,"owners_count":21999038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-neural-networks","conformal-prediction","deep-learning","earth-observation","predictive-uncertainty","pytorch","pytorch-lightning","uncertainty-quantification"],"created_at":"2024-09-25T01:03:48.099Z","updated_at":"2025-05-16T12:10:50.698Z","avatar_url":"https://github.com/lightning-uq-box.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://github.com/lightning-uq-box/lightning-uq-box/blob/main/docs/_static/lettering.jpeg?raw=true\" alt=\"Lightning-UQ-Box logo\" width=\"600\" height=\"auto\" /\u003e\n\u003c/p\u003e\n\n[![docs](https://readthedocs.org/projects/lightning-uq-box/badge/?version=latest)](https://lightning-uq-box.readthedocs.io/en/latest/)\n[![style](https://github.com/lightning-uq-box/lightning-uq-box/actions/workflows/style.yaml/badge.svg)](https://github.com/lightning-uq-box/lightning-uq-box/actions/workflows/style.yaml)\n[![tests](https://github.com/lightning-uq-box/lightning-uq-box/actions/workflows/tests.yaml/badge.svg)](https://github.com/lightning-uq-box/lightning-uq-box/actions/workflows/tests.yaml)\n[![codecov](https://codecov.io/gh/lightning-uq-box/lightning-uq-box/branch/main/graph/badge.svg?token=oa3Z3PMVOg)](https://app.codecov.io/gh/lightning-uq-box/lightning-uq-box)\n[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/lightning-uq-box/lightning-uq-box/blob/main/LICENSE)\n\u003ca href=\"https://pytorch.org/get-started/locally/\"\u003e\u003cimg alt=\"PyTorch\" src=\"https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href=\"https://pytorchlightning.ai/\"\u003e\u003cimg alt=\"Lightning\" src=\"https://img.shields.io/badge/-Lightning-792ee5?logo=pytorchlightning\u0026logoColor=white\"\u003e\u003c/a\u003e \u0026emsp;\n\n# lightning-uq-box\n\nThe lightning-uq-box is a PyTorch library that provides various Uncertainty Quantification (UQ) techniques for modern neural network architectures.\n\nWe hope to provide the starting point for a collaborative open source effort to make it easier for practitioners to include UQ in their workflows and\nremove possible barriers of entry. Additionally, we hope this can be a pathway to more easily compare methods across UQ frameworks and potentially enhance the development of new UQ methods for neural networks.\n\n*The project is currently under active development, but we nevertheless hope for early feedback, feature requests, or contributions. Please check the [Contribution Guide](https://lightning-uq-box.readthedocs.io/en/latest/contribute.html) for further information.*\n\nThe goal of this library is threefold:\n\n1. Provide implementations for a variety of Uncertainty Quantification methods for Modern Deep Neural Networks that work with a range of neural network architectures and have different theoretical underpinnings\n2. Make it easy to compare UQ methods on a given dataset\n3. Focus on reproducibility of experiments with minimum boiler plate code and standardized evaluation protocols\n\nTo this end, each UQ-Method is essentially just a [Lightning Module](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html) which can be used with a [Lightning Data Module](https://lightning.ai/docs/pytorch/stable/data/datamodule.html) and a [Trainer](https://lightning.ai/docs/pytorch/stable/common/trainer.html) to execute training, evaluation and inference for your desired task. The library also utilizes the [Lightning Command Line Interface (CLI)](https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.cli.LightningCLI.html) for better reproducibility of experiments and setting up experiments at scale.\n\n# Theory Guide\n\nFor a comprehensive document that provides more mathematical details for each method and generally forms the basis of our implementations, please see the [Theory Guide](./docs/api/Lightning_UQ_Box_Theory_Guide.pdf). As a living document, we plan to update it as the library encompasses more methods. If you have any questions, or find typos or errors, feel free to open an issue.\n\n# Installation\n\nThe recommended way to install the latest released version is via pip,\n\n```console\npip install lightning-uq-box\n```\n\nFor the latest development version you can run,\n\n```console\npip install git+https://github.com/lightning-uq-box/lightning-uq-box.git\n```\n\nThe package is also available for installation via conda or spack. You can find instructions in the [documention](https://lightning-uq-box.readthedocs.io/en/latest/installation.html)\n\n# UQ-Methods\n\nIn the tables that follow below, you can see what UQ-Method/Task combination is currently supported by the Lightning-UQ-Box via these indicators:\n\n- ✅ supported\n- ❌ not designed for this task\n- ⏳ in progress\n\nThe implemented methods are of course not exhaustive, as the number of new methods keeps increasing. For an overview of methods that we are tracking or are planning to support, take a look at [this issue](https://github.com/lightning-uq-box/lightning-uq-box/issues/43).\n\n## Classification of UQ-Methods\n\nThe following sections aims to give an overview of different UQ-Methods by grouping them according to some commonalities. We agree that there could be other groupings as well and welcome suggestions to improve this overview. We also follow this grouping for the API documentation in the hopes to make navigation easier.\n\n### Single Forward Pass Methods\n\n| UQ-Method                                     | Regression | Classification | Segmentation | Pixel Wise Regression |\n|-----------------------------------------------|:----------:|:--------------:|:------------:|:---------------------:|\n| Quantile Regression (QR)                      |     ✅     |       ❌       |      ❌      |          ✅           |\n| Deep Evidential (DE)                          |     ✅     |       ⏳       |      ⏳      |          ✅           |\n| Mean Variance Estimation (MVE)                |     ✅     |       ❌       |      ❌      |          ✅           |\n| ZigZag                                        |     ✅     |       ✅       |      ❌      |          ❌           |\n| Mixture Density Networks                      |     ✅     |       ❌       |      ❌      |          ⏳           |\n\n### Approximate Bayesian Methods\n\n| UQ-Method                                     | Regression | Classification | Segmentation | Pixel Wise Regression |\n|-----------------------------------------------|:----------:|:--------------:|:------------:|:---------------------:|\n| Bayesian Neural Network VI ELBO (BNN_VI_ELBO) |     ✅     |       ✅       |      ✅      |          ⏳           |\n| Bayesian Neural Network VI (BNN_VI)           |     ✅     |       ⏳       |      ⏳      |          ⏳           |\n| Deep Kernel Learning (DKL)                    |     ✅     |       ✅       |      ❌      |          ❌           |\n| Deterministic Uncertainty Estimation (DUE)    |     ✅     |       ✅       |      ❌      |          ❌           |\n| Laplace Approximation (Laplace)               |     ✅     |       ✅       |      ❌      |          ❌           |\n| Monte Carlo Dropout (MC-Dropout)              |     ✅     |       ✅       |      ✅      |          ✅           |\n| Stochastic Gradient Langevin Dynamics (SGLD)  |     ✅     |       ✅       |      ⏳      |          ⏳           |\n| Spectral Normalized Gaussian Process (SNGP)   |     ✅     |       ✅       |      ❌      |          ❌           |\n| Stochastic Weight Averaging Gaussian (SWAG)   |     ✅     |       ✅       |      ✅      |          ✅           |\n| Variational Bayesian Last Layer (VBLL)        |     ✅     |       ✅       |      ❌      |          ❌           |\n| Deep Ensemble                                 |     ✅     |       ✅       |      ✅      |          ✅           |\n| Masked Ensemble                               |     ✅     |       ✅       |      ⏳      |          ⏳           |\n| Density Uncertainty Layer                     |     ✅     |       ✅       |      ❌      |          ❌           |\n\n### Generative Models\n\n| UQ-Method                                     | Regression | Classification | Segmentation | Pixel Wise Regression |\n|-----------------------------------------------|:----------:|:--------------:|:------------:|:---------------------:|\n| Classification And Regression Diffusion (CARD)|     ✅     |       ✅       |      ❌      |          ❌           |\n| Probabilistic UNet                            |     ❌     |       ❌       |      ✅      |          ❌           |\n| Hierarchical Probabilistic UNet               |     ❌     |       ❌       |      ✅      |          ❌           |\n| Variational Auto-Encoder (VAE)                |     ❌     |       ❌       |      ❌      |          ✅           |\n\n### Post-Hoc methods\n\n| UQ-Method                                     | Regression | Classification | Segmentation | Pixel Wise Regression |\n|-----------------------------------------------|:----------:|:--------------:|:------------:|:---------------------:|\n| Test Time Augmentation (TTA)                  |     ✅     |       ✅       |      ⏳      |          ⏳           |\n| Temperature Scaling                           |     ❌     |       ✅       |      ⏳      |          ❌           |\n| Conformal Quantile Regression (Conformal QR)  |     ✅     |       ❌       |      ❌      |          ⏳           |\n| Regularized Adaptive Prediction Sets (RAPS)   |     ❌     |       ✅       |      ❌      |          ❌           |\n| Image to Image Conformal                      |     ❌     |       ❌       |      ❌      |          ✅           |\n\n# Tutorials\n\nWe try to provide many different tutorials so that users can get a better understanding of implemented methods and get a feel for how they apply to different problems.\nHead over to the [tutorials](https://lightning-uq-box.readthedocs.io/en/latest/tutorial_overview.html) page to get started. These tutorials can also be launched in google colab if you navigate to the rocket icon at the top of a tutorial page.\n\n# Documentation\nWe aim to provide an extensive documentation on all included UQ-methods that provide some theoretical background, as well as tutorials that illustrate these methods on toy datasets.\n\n# Citation\n\nIf you use this software in your work, please cite our [paper](https://jmlr.org/papers/v26/24-2110.html):\n\n```bibtex\n@article{JMLR:v26:24-2110,\n  author  = {Nils Lehmann and Nina Maria Gottschling and Jakob Gawlikowski and Adam J. Stewart and Stefan Depeweg and Eric Nalisnick},\n  title   = {Lightning UQ Box: Uncertainty Quantification for Neural Networks},\n  journal = {Journal of Machine Learning Research},\n  year    = {2025},\n  volume  = {26},\n  number  = {54},\n  pages   = {1--7},\n  url     = {http://jmlr.org/papers/v26/24-2110.html}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-uq-box%2Flightning-uq-box","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flightning-uq-box%2Flightning-uq-box","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightning-uq-box%2Flightning-uq-box/lists"}