{"id":29625204,"url":"https://github.com/novartis/unique","last_synced_at":"2025-07-21T06:07:38.204Z","repository":{"id":254304911,"uuid":"844742423","full_name":"Novartis/UNIQUE","owner":"Novartis","description":"A Python library for benchmarking uncertainty estimation and quantification methods for Machine Learning models predictions.","archived":false,"fork":false,"pushed_at":"2024-10-04T18:16:59.000Z","size":6290,"stargazers_count":37,"open_issues_count":0,"forks_count":4,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-06-29T12:16:42.982Z","etag":null,"topics":["calibration","machine-learning","uncertainty-quantification"],"latest_commit_sha":null,"homepage":"https://opensource.nibr.com/UNIQUE","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Novartis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-19T22:09:07.000Z","updated_at":"2025-06-26T07:44:38.000Z","dependencies_parsed_at":"2024-08-22T17:45:09.677Z","dependency_job_id":"7225e1df-77bb-4a45-991c-a8c167c3b9cd","html_url":"https://github.com/Novartis/UNIQUE","commit_stats":null,"previous_names":["novartis/unique"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Novartis/UNIQUE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Novartis%2FUNIQUE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Novartis%2FUNIQUE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Novartis%2FUNIQUE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Novartis%2FUNIQUE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Novartis","download_url":"https://codeload.github.com/Novartis/UNIQUE/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Novartis%2FUNIQUE/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266248501,"owners_count":23899056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["calibration","machine-learning","uncertainty-quantification"],"created_at":"2025-07-21T06:07:37.659Z","updated_at":"2025-07-21T06:07:38.188Z","avatar_url":"https://github.com/Novartis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cfigure\u003e\n  \u003cimg src=\"https://github.com/Novartis/UNIQUE/raw/main/docs/source/_static/unique_logo_blue.png\" alt=\"UNIQUE Logo\"\u003e\n  \u003cfigcaption align=center\u003e\u003cu\u003e\u003cb\u003eUN\u003c/b\u003e\u003c/u\u003ecerta\u003cu\u003e\u003cb\u003eI\u003c/b\u003e\u003c/u\u003enty \u003cu\u003e\u003cb\u003eQU\u003c/b\u003e\u003c/u\u003eantification b\u003cu\u003e\u003cb\u003eE\u003c/b\u003e\u003c/u\u003enchmark: a Python library for benchmarking uncertainty estimation and quantification methods for Machine Learning models predictions.\u003c/figcaption\u003e\n\u003c/figure\u003e\n\n[![Python](https://img.shields.io/pypi/pyversions/unique-uncertainty?label=Python)](https://pypi.org/project/unique-uncertainty/)\n[![PyPI version](https://img.shields.io/pypi/v/unique-uncertainty?color=green\u0026label=PyPI)](https://pypi.org/project/unique-uncertainty/)\n[![Conda version](https://img.shields.io/conda/vn/conda-forge/unique-uncertainty?color=green\u0026label=conda-forge)](https://anaconda.org/conda-forge/unique-uncertainty)\n[![License](https://img.shields.io/badge/License-BSD_3--Clause-red)](https://opensource.org/licenses/BSD-3-Clause)\n[![chemRxiv](https://img.shields.io/badge/chemRxiv-10.26434%2Fchemrxiv--2024--fmbgk-yellow)](https://doi.org/10.26434/chemrxiv-2024-fmbgk)\n[![PyPI downloads](https://img.shields.io/pypi/dm/unique-uncertainty?color=yellowgreen\u0026label=PyPI%20downloads)](https://pypi.org/project/unique-uncertainty/)\n[![Conda downloads](https://img.shields.io/conda/dn/conda-forge/unique-uncertainty?color=yellowgreen\u0026label=conda%20downloads)](https://anaconda.org/conda-forge/unique-uncertainty)\n[![Docs](https://github.com/Novartis/UNIQUE/actions/workflows/docs.yml/badge.svg?branch=main)](https://opensource.nibr.com/UNIQUE/)\n[![Build](https://github.com/Novartis/UNIQUE/actions/workflows/build.yml/badge.svg?branch=main)](https://pypi.org/project/unique-uncertainty/)\n\n\n## Introduction\n\n`UNIQUE` provides methods for quantifying and evaluating the uncertainty of Machine Learning (ML) models predictions. The library allows to combine and benchmark multiple uncertainty quantification (UQ) methods simultaneously, generates intuitive visualizations, evaluates the goodness of the UQ methods against established metrics, and in general enables the users to get a comprehensive overview of their ML model's performances from an uncertainty quantification perspective.\n\n`UNIQUE` is a model-agnostic tool, meaning that it does not depend on any specific ML model building platform or provides any  ML model training functionality. It is lightweight, because it only requires the user to input their model's inputs and predictions.\n\n\u003cfigure\u003e\n  \u003cimg src=\"https://github.com/Novartis/UNIQUE/raw/main/docs/source/_static/schema_high_level.png\" alt=\"UNIQUE High Level Schema\"\u003e\n  \u003cfigcaption align=center\u003eHigh-level schema of \u003ccode\u003eUNIQUE\u003c/code\u003e's components.\u003c/figcaption\u003e\n\u003c/figure\u003e\n\n\n## Installation\n\n[![Python](https://img.shields.io/pypi/pyversions/unique-uncertainty?label=Python)](https://pypi.org/project/unique-uncertainty/)\n[![PyPI version](https://img.shields.io/pypi/v/unique-uncertainty?color=green\u0026label=PyPI)](https://pypi.org/project/unique-uncertainty/)\n[![Conda version](https://img.shields.io/conda/vn/conda-forge/unique-uncertainty?color=green\u0026label=conda-forge)](https://anaconda.org/conda-forge/unique-uncertainty)\n[![PyPI downloads](https://img.shields.io/pypi/dm/unique-uncertainty?color=yellowgreen\u0026label=PyPI%20downloads)](https://pypi.org/project/unique-uncertainty/)\n[![Conda downloads](https://img.shields.io/conda/dn/conda-forge/unique-uncertainty?color=yellowgreen\u0026label=conda%20downloads)](https://anaconda.org/conda-forge/unique-uncertainty)\n[![Build](https://github.com/Novartis/UNIQUE/actions/workflows/build.yml/badge.svg?branch=main)](https://pypi.org/project/unique-uncertainty/)\n\n`UNIQUE` is currently compatible with Python 3.8 through 3.12.1. To install the latest release and use the package as is, run the following in a compatible environment of choice:\n\n```bash\npip install unique-uncertainty\n```\n\nor:\n\n```bash\nconda install -c conda-forge unique-uncertainty\n# mamba install -c conda-forge unique-uncertainty\n```\n\nCheck out the [docs](https://opensource.nibr.com/UNIQUE/installation.html#installation) for more installation instructions.\n\n\n## Getting Started\n\nCheck out the [docs](https://opensource.nibr.com/UNIQUE/getting_started/index.html#getting-started) for a complete set of instructions on how to prepare your data and the possible configurations offered by `UNIQUE`.\n\n\n## Usage\n\nFinally, once the data and configuration files have been prepared, you can run `UNIQUE` in the following way:\n\n```python\nfrom unique import Pipeline\n\n# Prepare UNIQUE pipeline\npipeline = Pipeline.from_config(\"/path/to/config.yaml\")\n\n# Run UNIQUE pipeline\nuq_methods_outputs, uq_evaluation_outputs = pipeline.fit()\n# Returns: (Dict[str, np.ndarray], Dict[str, pd.DataFrame])\n```\n\nFitting the `Pipeline` will return two dictionaries:\n\n- `uq_methods_outputs`: contains each UQ method's name (as in \"UQ_Method_Name[Input_Name(s)]\") and computed UQ values.\n- `uq_evaluation_outputs`: contains, for each evaluation type (ranking-based, proper scoring rules, and calibration-based), the evaluation metrics outputs for all the corresponding UQ methods organized in `pd.DataFrame`.\n\nAdditionally, `UNIQUE` also generates graphical outputs in the form of tables and evaluation plots (if `display_outputs` is enabled and the code is running in a JupyterNotebook cell).\n\n\n### Examples\n\nFor more hands-on examples and detailed usage, check out some of the examples in the [docs](https://opensource.nibr.com/UNIQUE/examples/index.html#examples).\n\n\n## Deep Dive\n\nCheck out the [docs](https://opensource.nibr.com/UNIQUE/indepth/index.html#deep-dive) for an in-depth overview of `UNIQUE`'s concepts, functionalities, outputs, and references.\n\n\n## Contributing\n\nAny and all contributions and suggestions from the community are more than welcome and highly appreciated. If you wish to help us out in making `UNIQUE` even better, please check out our [contributing guidelines](./CONTRIBUTING.md).\n\nPlease note that we have a [Code of Conduct](./CODE_OF_CONDUCT.md) in place to ensure a positive and inclusive community environment. By participating in this project, you agree to abide by its terms.\n\n\n## License\n\n[![License](https://img.shields.io/badge/License-BSD_3--Clause-red)](https://opensource.org/licenses/BSD-3-Clause)\n\n`UNIQUE` is licensed under the BSD 3-Clause License. See the [LICENSE](./LICENSE.md) file.\n\n\n## Cite Us\n\n[![chemRxiv](https://img.shields.io/badge/chemRxiv-10.26434%2Fchemrxiv--2024--fmbgk-yellow)](https://doi.org/10.26434/chemrxiv-2024-fmbgk)\n\nIf you find `UNIQUE` helpful for your work and/or research, please consider citing our work:\n\n```bibtex\n@misc{lanini2024unique,\n  title={UNIQUE: A Framework for Uncertainty Quantification Benchmarking},\n  author={Lanini, Jessica and Huynh, Minh Tam Davide and Scebba, Gaetano and Schneider, Nadine and Rodr{\\'\\i}guez-P{\\'e}rez, Raquel},\n  year={2024},\n  doi={https://doi.org/10.26434/chemrxiv-2024-fmbgk},\n}\n```\n\n\n## Contacts \u0026 Acknowledgements\n\nFor any questions or further details about the project, please get in touch with any of the following contacts:\n\n* **[Jessica Lanini](mailto:jessica.lanini@novartis.com?subject=UNIQUE)**\n* **[Minh Tam Davide Huynh](https://github.com/mtdhuynh)**\n* **[Gaetano Scebba](mailto:gaetano.scebba@novartis.com?subject=UNIQUE)**\n* **[Nadine Schneider](mailto:nadine-1.schneider@novartis.com?subject=UNIQUE)**\n* **[Raquel Rodríguez-Pérez](mailto:raquel.rodriguez_perez@novartis.com?subject=UNIQUE)**\n\n\n![Novartis Logo](https://github.com/Novartis/UNIQUE/raw/main/docs/source/_static/novartis_logo.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnovartis%2Funique","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnovartis%2Funique","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnovartis%2Funique/lists"}