{"id":13688852,"url":"https://github.com/deel-ai/puncc","last_synced_at":"2026-04-09T15:04:05.143Z","repository":{"id":146320727,"uuid":"515914540","full_name":"deel-ai/puncc","owner":"deel-ai","description":"👋 Puncc is a python library for predictive uncertainty quantification using conformal prediction.","archived":false,"fork":false,"pushed_at":"2025-01-10T18:10:50.000Z","size":14484,"stargazers_count":325,"open_issues_count":4,"forks_count":17,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-01T20:37:07.116Z","etag":null,"topics":["conformal-inference","conformal-prediction","conformal-regressors","uncertainty-estimation","uncertainty-quantification"],"latest_commit_sha":null,"homepage":"https://deel-ai.github.io/puncc/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deel-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSES/headers/MIT-Clause.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":"GOVERNANCE.md","roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-07-20T09:22:41.000Z","updated_at":"2025-04-23T09:51:18.000Z","dependencies_parsed_at":"2024-03-20T15:45:28.894Z","dependency_job_id":"ddce7601-8418-4418-9822-d6c2effb0af2","html_url":"https://github.com/deel-ai/puncc","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deel-ai%2Fpuncc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deel-ai%2Fpuncc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deel-ai%2Fpuncc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deel-ai%2Fpuncc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deel-ai","download_url":"https://codeload.github.com/deel-ai/puncc/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254442819,"owners_count":22071878,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conformal-inference","conformal-prediction","conformal-regressors","uncertainty-estimation","uncertainty-quantification"],"created_at":"2024-08-02T15:01:25.054Z","updated_at":"2026-04-09T15:04:05.136Z","avatar_url":"https://github.com/deel-ai.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003c!-- Banner --\u003e\n\u003cdiv align=\"center\"\u003e\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"docs/assets/banner_dark.png\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"docs/assets/banner_light.png\"\u003e\n  \u003cimg src=\"docs/assets/banner_light.png\" alt=\"Puncc\" width=\"90%\" align=\"right\"\u003e\n\u003c/picture\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n\u003c!-- Badges --\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"#\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Python-3.9 +-efefef\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"#\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/License-MIT-efefef\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/deel-ai/puncc/actions/workflows/linter.yml\"\u003e\n    \u003cimg alt=\"PyLint\" src=\"https://github.com/deel-ai/puncc/actions/workflows/linter.yml/badge.svg\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/deel-ai/puncc/actions/workflows/tests.yml\"\u003e\n    \u003cimg alt=\"Tox\" src=\"https://github.com/deel-ai/puncc/actions/workflows/tests.yml/badge.svg\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/deel-ai/puncc/actions/workflows/tests-coverage.yml\"\u003e\n    \u003cimg alt=\"Coverage\" src=\"https://raw.githubusercontent.com/deel-ai/puncc/badges/.github/badges/coverage.svg\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n***Puncc*** (short for **P**redictive **un**certainty **c**alibration and **c**onformalization) is an open-source Python library. It seamlessly integrates a collection of state-of-the-art conformal prediction algorithms and associated techniques for diverse machine learning tasks, including regression, classification, object detection and anomaly detection.\n***Puncc*** can be used with any predictive model to provide rigorous uncertainty estimations.\nUnder data exchangeability (or *i.i.d*), the generated prediction sets are guaranteed to cover the true outputs within a user-defined error $\\alpha$.\n\nDocumentation is available [**online**](https://deel-ai.github.io/puncc/index.html).\n\n## 📚 Table of contents\n\n- [🐾 Installation](#-installation)\n- [📖 Documentation](#-documentation)\n- [👨‍🎓 Tutorials](#-tutorials)\n- [🚀 QuickStart](#-quickstart)\n- [📚 Citation](#-citation)\n- [💻 Contributing](#-contributing)\n- [🙏 Acknowledgments](#-acknowledgments)\n- [👨‍💻 Creators](#-creators)\n- [📝 License](#-license)\n\n## 🐾 Installation\n\n*puncc* requires a version of python higher than 3.9 and several libraries including Scikit-learn. It is recommended to install *puncc* in a virtual environment to not mess with your system's dependencies.\n\nYou can directly install the library using pip:\n\n```bash\npip install -U puncc\n```\n\nThe package is also available on conda-forge:\n\n```bash\nconda install -c conda-forge puncc\n```\n\n## 📖 Documentation\n\nFor comprehensive documentation, we encourage you to visit the [**official documentation page**](https://deel-ai.github.io/puncc/index.html).\n\n\u003ca id=\"-tutorials\"\u003e\u003c/a\u003e\n## 👨‍🎓 Tutorials\n\nWe highly recommend following the introductory tutorials to get familiar with the library and its API.\n\n| Tutorial | Description | Link |\n|----------|-------------|------|\n| **Introduction Tutorial** | Get started with the basics of *puncc*. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_intro.ipynb)  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_intro.ipynb) |\n| **API Tutorial** | Learn about *puncc*'s API. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/api_intro.ipynb)  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/api_intro.ipynb) |\n| **Tutorial on CP with PyTorch** | Learn how to use *puncc* with PyTorch. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_pytorch.ipynb)  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_pytorch.ipynb) |\n| **Adaptive Conformal Regression** | Learn how conformalize regression models to obtain adaptive prediction intervals using methods such as LACP, LWCP and CQR. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_adaptive_methods.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_adaptive_methods.ipynb) |\n| **Conformal Object Detection** | Learn to conformalize an object detector. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_cod.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_cod.ipynb) |\n| **Conformal Text Classification** | Learn to conformalize pretrained HuggingFace NLP model. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_nlp.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_nlp.ipynb) |\n| **Conformal Classwise Classification** | Learn how to conformalize classification models for class-conditional coverage. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_classwise.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_classwise.ipynb) |\n| **Conformal Time Series Forecasting** | Learn to conformalize time series models. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_timeseries.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_timeseries.ipynb) |\n| **Architecture Overview** | Detailed overview of *puncc*'s architecture. | [![Open In Github](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white)](docs/puncc_architecture.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/deel-ai/puncc/blob/main/docs/puncc_architecture.ipynb) |\n\n## 🚀 Quickstart\n\nConformal prediction enables to transform point predictions into interval predictions with high probability of coverage. The figure below shows the result of applying the split conformal algorithm on a linear regressor.\n\n\u003cfigure style=\"text-align:center\"\u003e\n\u003cimg src=\"docs/assets/cp_process.png\"/\u003e\n\u003c/figure\u003e\n\nMany conformal prediction algorithms can easily be applied using *puncc*.  The code snippet below shows the example of split conformal prediction with a pretrained linear model:\n\n ```python\n from deel.puncc.api.prediction import BasePredictor\nfrom deel.puncc.regression import SplitCP\n\n# Load calibration and test data\n# ...\n\n# Pretrained regression model\n# trained_linear_model = ...\n\n# Wrap the model to enable interoperability with different ML libraries\ntrained_predictor =  BasePredictor(trained_linear_model)\n\n# Instanciate the split conformal wrapper for the linear model.\n# Train argument is set to False because we do not want to retrain the model\nsplit_cp = SplitCP(trained_predictor, train=False)\n\n# With a calibration dataset, compute (and store) nonconformity scores\nsplit_cp.fit(X_calib=X_calib, y_calib=y_calib)\n\n# Obtain the model's point prediction y_pred and prediction interval\n# PI = [y_pred_lower, y_pred_upper] for a target coverage of 90% (1-alpha).\ny_pred, y_pred_lower, y_pred_upper = split_cp.predict(X_test, alpha=0.1)\n```\n\n\nThe library provides several metrics (`deel.puncc.metrics`) and plotting capabilities (`deel.puncc.plotting`) to evaluate and visualize the results of a conformal procedure. For a target error rate of $\\alpha = 0.1$, the marginal coverage reached in this example on the test set is higher than $90$% (see [**Introduction tutorial**](docs/puncc_intro.ipynb)):\n\u003cdiv align=\"center\"\u003e\n\u003cfigure style=\"text-align:center\"\u003e\n\u003cimg src=\"docs/assets/results_quickstart_split_cp_pi.png\" alt=\"90% Prediction Interval with the Split Conformal Prediction Method\" width=\"70%\"/\u003e\n\u003cdiv align=center\u003e90% Prediction Interval with Split Conformal Prediction.\u003c/div\u003e\n\u003c/figure\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n### More flexibility with the API\n\n*Puncc* provides two ways of defining and using conformal prediction wrappers:\n- A direct approach to run state-of-the-art conformal prediction procedures. This is what we used in the previous conformal regression example.\n- **Low-level API**: a more flexible approach based of full customization of the prediction model, the choice of nonconformity scores and the split between fit and calibration datasets.\n\nA quick comparison of both approaches is provided in the [**API tutorial**](docs/api_intro.ipynb) for a regression problem.\n\n\u003cfigure style=\"text-align:center\"\u003e\n    \u003cimg src=\"docs/assets/puncc_architecture.png\" width=\"100%\"/\u003e\n\u003c/figure\u003e\n\n### 🖥️ Implemented Algorithms\n\u003cdetails open\u003e\n  \u003csummary\u003eOverview of Implemented Methods from the Literature:\u003c/summary\u003e\n\n| Procedure Type                          | Procedure Name                                        | Description (more details in [Theory overview](https://deel-ai.github.io/puncc/theory_overview.html))                |\n|-----------------------------------------|------------------------------------------------------|-------------------------------------------------------|\n| Conformal Regression                    | [`deel.puncc.regression.SplitCP`](https://deel-ai.github.io/puncc/regression.html#deel.puncc.regression.SplitCP)                      | Split Conformal Regression                            |\n| Conformal Regression                    | [`deel.puncc.regression.LocallyAdaptiveCP`](https://deel-ai.github.io/puncc/regression.html#deel.puncc.regression.LocallyAdaptiveCP)            | Locally Adaptive Conformal Regression                 |\n| Conformal Regression                    | [`deel.puncc.regression.LeverageWeightedCP`](https://deel-ai.github.io/puncc/regression.html#deel.puncc.regression.LeverageWeightedCP)            | Leverage-Weighted Conformal Prediction                 |\n| Conformal Regression                    | [`deel.puncc.regression.CQR`](https://deel-ai.github.io/puncc/regression.html#deel.puncc.regression.CQR)                         | Conformalized Quantile Regression                     |\n| Conformal Regression                    | [`deel.puncc.regression.CvPlus`](https://deel-ai.github.io/puncc/regression.html#deel.puncc.regression.CVPlus)                       | CV + (cross-validation)                               |\n| Conformal Regression                    | [`deel.puncc.regression.EnbPI`](https://deel-ai.github.io/puncc/regression.html#deel.puncc.regression.EnbPI)                        | Ensemble Batch Prediction Intervals method            |\n| Conformal Regression                    | [`deel.puncc.regression.aEnbPI`](https://deel-ai.github.io/puncc/regression.html#deel.puncc.regression.AdaptiveEnbPI)                       | Locally adaptive Ensemble Batch Prediction Intervals method |\n| Conformal Classification                | [`deel.puncc.classification.LAC`](https://deel-ai.github.io/puncc/classification.html#deel.puncc.classification.LAC)                      | Least Ambiguous Set-Valued Classifiers                |\n| Conformal Classification                | [`deel.puncc.classification.ClasswiseLAC`](https://deel-ai.github.io/puncc/classification.html#deel.puncc.classification.ClasswiseLAC)                      | Classwise Conformal Classification                    |\n| Conformal Classification                | [`deel.puncc.classification.APS`](https://deel-ai.github.io/puncc/classification.html#deel.puncc.classification.APS)                      | Adaptive Prediction Sets                              |\n| Conformal Classification                | [`deel.puncc.classification.RAPS`](https://deel-ai.github.io/puncc/classification.html#deel.puncc.classification.RAPS)                     | Regularized Adaptive Prediction Sets (APS is a special case where $\\lambda = 0$) |\n| Conformal Anomaly Detection             | [`deel.puncc.anomaly_detection.SplitCAD`](https://deel-ai.github.io/puncc/anomaly_detection.html#deel.puncc.anomaly_detection.SplitCAD)              | Split Conformal Anomaly detection (used to control the maximum false positive rate) |\n| Conformal Object Detection              | [`deel.puncc.object_detection.SplitBoxWise`](https://deel-ai.github.io/puncc/object_detection.html#deel.puncc.object_detection.SplitBoxWise)           | Box-wise split conformal object detection             |\n\n\u003c/details\u003e\n\n## 📚 Citation\n\nIf you use our library for your work, please cite our paper:\n\n```\n@inproceedings{mendil2023puncc,\n  title={PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization},\n  author={Mendil, Mouhcine and Mossina, Luca and Vigouroux, David},\n  booktitle={Conformal and Probabilistic Prediction with Applications},\n  pages={582--601},\n  year={2023},\n  organization={PMLR}\n}\n```\n\n*Puncc* has been used to support the work presented in our COPA 2022 paper on conformal prediction for time series.\n\n```\n@inproceedings{mendil2022robust,\n  title={Robust Gas Demand Forecasting With Conformal Prediction},\n  author={Mendil, Mouhcine and Mossina, Luca and Nabhan, Marc and Pasini, Kevin},\n  booktitle={Conformal and Probabilistic Prediction with Applications},\n  pages={169--187},\n  year={2022},\n  organization={PMLR}\n}\n```\n\n\n## 🙏 Acknowledgments\n\n\u003cimg align=\"right\" src=\"https://www.irt-saintexupery.com/wp-content/uploads/logo-DEEL.png\" width=\"25%\"\u003e\nThis project received funding from the French ”Investing for the Future – PIA3” program within the Artificial and Natural Intelligence Toulouse Institute (ANITI). The authors gratefully acknowledge the support of the \u003ca href=\"https://www.irt-saintexupery.com/deel/\"\u003e DEEL \u003c/a\u003e project.\n\n## 👨‍💻 About the Developers\n\nPuncc's development team is a group of passionate scientists and engineers who are committed to developing a dependable and user-friendly open-source software. We are always looking for new contributors to this initiative. If you are interested in helping us develop puncc, please feel free to get involved.\n\n## 💻 Contributing\n\nContributions are welcome! Feel free to report an issue or open a pull\nrequest. Take a look at our guidelines [here](CONTRIBUTING.md).\n\n## 🔑 License\n\nThe package is released under [MIT](LICENSES/headers/MIT-Clause.txt) license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeel-ai%2Fpuncc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeel-ai%2Fpuncc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeel-ai%2Fpuncc/lists"}