{"id":25058489,"url":"https://github.com/jordandeklerk/pyloo","last_synced_at":"2026-03-05T01:02:02.894Z","repository":{"id":274903874,"uuid":"924385364","full_name":"jordandeklerk/pyloo","owner":"jordandeklerk","description":"Python package for approximate leave-one-out cross-validation (LOO-CV) and Pareto smoothed importance sampling (PSIS) for Bayesian Modeling","archived":false,"fork":false,"pushed_at":"2025-05-26T01:16:49.000Z","size":4654,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-26T02:32:41.700Z","etag":null,"topics":["bayes","bayesian","bayesian-data-analysis","bayesian-inference","bayesian-methods","bayesian-statistics","cross-validation","information-criterion","model-comparison","python","python-package"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jordandeklerk.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-29T22:43:19.000Z","updated_at":"2025-05-26T01:16:50.000Z","dependencies_parsed_at":"2025-02-13T06:25:59.377Z","dependency_job_id":"6040f179-8f9c-418c-8755-fc01fa019eaf","html_url":"https://github.com/jordandeklerk/pyloo","commit_stats":null,"previous_names":["jordandeklerk/pyloo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jordandeklerk/pyloo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jordandeklerk%2Fpyloo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jordandeklerk%2Fpyloo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jordandeklerk%2Fpyloo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jordandeklerk%2Fpyloo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jordandeklerk","download_url":"https://codeload.github.com/jordandeklerk/pyloo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jordandeklerk%2Fpyloo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30104218,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T00:38:46.881Z","status":"ssl_error","status_checked_at":"2026-03-05T00:38:45.829Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayes","bayesian","bayesian-data-analysis","bayesian-inference","bayesian-methods","bayesian-statistics","cross-validation","information-criterion","model-comparison","python","python-package"],"created_at":"2025-02-06T14:35:21.580Z","updated_at":"2026-03-05T01:02:02.881Z","avatar_url":"https://github.com/jordandeklerk.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"assets/pyloo-dark.png#gh-light-mode-only\" width=\"175\" align=\"left\" alt=\"sieve logo\"\u003e\u003c/img\u003e\n\u003cimg src=\"assets/pyloo-light.png#gh-dark-mode-only\" width=\"175\" align=\"left\" alt=\"sieve logo\"\u003e\u003c/img\u003e\n\n[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](https://www.repostatus.org/badges/latest/active.svg)](https://www.repostatus.org/#active)\n[![Build Status](https://github.com/jordandeklerk/pyloo/actions/workflows/test.yml/badge.svg)](https://github.com/jordandeklerk/pyloo/actions/workflows/test.yml)\n[![Code Coverage](https://codecov.io/gh/jordandeklerk/pyloo/branch/main/graph/badge.svg)](https://codecov.io/gh/jordandeklerk/pyloo)\n[![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)\n[![Type Checked](https://img.shields.io/badge/type--checked-mypy-blue?style=flat-square\u0026logo=python)](https://mypy-lang.org/)\n[![Last commit](https://img.shields.io/github/last-commit/jordandeklerk/pyloo)](https://github.com/jordandeklerk/pyloo/graphs/commit-activity)\n[![Commit activity](https://img.shields.io/github/commit-activity/m/jordandeklerk/pyloo)](https://github.com/jordandeklerk/pyloo/graphs/commit-activity)\n[![Python version](https://img.shields.io/badge/3.11%20%7C%203.12%20%7C%203.13-blue?logo=python\u0026logoColor=white)](https://www.python.org/)\n\n__pyloo__ is a Python package providing efficient approximate leave-one-out cross-validation (LOO-CV) for fitted Bayesian models with advanced features for **PyMC** models. Inspired by its R twin [loo](https://github.com/stan-dev/loo), __pyloo__ brings similar functionality to the Python ecosystem.\n\u003cbr\u003e\u003cbr\u003e\u003cbr\u003e\n\n\u003e [!WARNING]\n\u003e Most of the ongoing development for *pyloo* is moving to the new **[arviz-stats](https://github.com/arviz-devs/arviz-stats)** project. Future enhancements and bug-fixes will be concentrated there, while this repository will remain as a lightweight wrapper and archival reference.\n\nThe package implements the fast and stable computations for approximate LOO-CV from\n\n* Vehtari, A., Gelman, A., and Gabry, J. (2017). Practical Bayesian model\nevaluation using leave-one-out cross-validation and WAIC.\n_Statistics and Computing_. 27(5), 1413--1432.\ndoi:10.1007/s11222-016-9696-4. [Online](https://link.springer.com/article/10.1007/s11222-016-9696-4),\n[arXiv preprint arXiv:1507.04544](https://arxiv.org/abs/1507.04544).\n\nand computes model weights as described in\n\n* Yao, Y., Vehtari, A., Simpson, D., and Gelman, A. (2018). Using\nstacking to average Bayesian predictive distributions. In Bayesian\nAnalysis, doi:10.1214/17-BA1091.\n[Online](https://projecteuclid.org/euclid.ba/1516093227),\n[arXiv preprint arXiv:1704.02030](https://arxiv.org/abs/1704.02030).\n\nWe recommend PSIS-LOO-CV over WAIC because PSIS offers informative diagnostics (like Pareto k estimates) and estimates for effective sample size and Monte Carlo error, providing greater insight into the reliability of the results.\n\n## Features\n\n### Core Functionality\n\n*   **Approximate Leave-One-Out Cross-Validation (LOO-CV)**: Efficiently compute LOO-CV using Pareto Smoothed Importance Sampling (PSIS) and other methods, complete with diagnostics (`loo`).\n*   **Model Comparison**: Compare models based on ELPD using various methods like stacking or Bayesian bootstrap (`loo_compare`).\n*   **LOO-Based Metrics**: Estimate predictive performance (e.g., MAE, MSE, CRPS) using LOO estimates (`loo_predictive_metric`, `loo_score`).\n*   **Non-factorized LOO-CV**: Compute LOO-CV for multivariate normal and Student-t models where the likelihood cannot be factorized by observations (`loo_nonfactor`).\n*   **Grouped \u0026 Subsampled LOO-CV**: Perform Leave-One-Group-Out CV (`loo_group`) and efficient subsampling for large datasets (`loo_subsample`).\n*   **Widely Applicable Information Criterion (WAIC)**: Calculate WAIC as an alternative model assessment metric (`waic`).\n\n### Advanced \u0026 PyMC Integration\n\n*   **PyMC Model Interface**: Seamless integration with PyMC models for streamlined workflow (`PyMCWrapper`).\n*   **Moment Matching**: Improve LOO-CV reliability for challenging observations by matching moments (`loo_moment_match`).\n*   **Exact Refitting (Reloo)**: Refit models for problematic observations identified by LOO diagnostics (`reloo`).\n*   **K-Fold Cross-Validation**: Flexible K-fold CV implementation with stratification, groups, and diagnostics (`loo_kfold`).\n*   **Variational Inference Support**: Compute LOO-CV for models fitted with Laplace or ADVI variational approximations (`loo_approximate_posterior`).\n\n## Quickstart\n\n### PSIS-LOO-CV\n\n```python\nimport pyloo as pl\nimport arviz as az\n\ndata = az.load_arviz_data(\"centered_eight\")\n\n# PSIS-LOO-CV\nloo_result = pl.loo(\n    data,\n    pointwise=True,  # Return pointwise values\n    method=\"psis\"    # Use PSIS (recommended)\n)\n\nprint(loo_result)\n```\n```\nComputed from 2000 posterior samples and 8 observations log-likelihood matrix.\n\n         Estimate       SE\nelpd_loo   -30.78      1.35\np_loo       0.95        0.48\nlooic      61.56       2.69\n\nAll Pareto k estimates are good (k \u003c 0.7).\nSee help('pareto-k-diagnostic') for details.\n```\n\n### Model Comparison\n\nCompare multiple models with `compare` using stacking weights or other methods:\n\n```python\nmodel1 = az.load_arviz_data(\"centered_eight\")\nmodel2 = az.load_arviz_data(\"non_centered_eight\")\n\ncomparison = pl.loo_compare(\n    {\n        \"centered\": model1,\n        \"non_centered\": model2\n    },\n    ic=\"loo\",                # Information criterion to use\n    method=\"stacking\",       # Method for computing weights\n    scale=\"log\"              # Scale for the scores\n)\n\nprint(comparison)\n```\n```\nModel comparison using LOO (scale: log)\n\n                elpd_loo   se      p_loo   weight    elpd_diff    dse     warning\nnon_centered     -30.72   1.33     0.90    1.00e+00     0.0      0.00      True\ncentered         -30.78   1.35     0.95    2.50e-16    -0.06     0.06      False\n\nAll Pareto k estimates are good (k \u003c 0.7)\n```\n\n## Installation\n\n\u003e ⚠️ **Note**: `pyloo` is currently under active development and has not yet been officially released or published to PyPI. While the core functionality is being stabilized, expect potential changes to the API and features. Installation is currently only possible directly from the development version on GitHub (see below).\n\n```bash\npip install pyloo\n```\n\nOr with conda:\n\n```bash\nconda install -c conda-forge pyloo\n```\n\n### Development Version\n\nTo install the latest development version directly from GitHub:\n\n```bash\npip install git+https://github.com/jordandeklerk/pyloo.git\n```\n\nFor development purposes, you can clone the repository and install in editable mode:\n\n```bash\ngit clone https://github.com/jordandeklerk/pyloo.git\ncd pyloo\npip install -e .\n```\n\n## Resources\n\n\u003e ⚠️ **Note**: Documentation coming soon.\n\n* [Documentation]() (API reference, examples)\n* [Open an issue]() (Bug reports, feature requests)\n\n## Citation\n\n```bibtex\n@article{vehtari2024practical,\n  title={Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC},\n  author={Vehtari, Aki and Gelman, Andrew and Gabry, Jonah},\n  journal={Statistics and Computing},\n  volume={27},\n  number={5},\n  pages={1413--1432},\n  year={2017},\n  publisher={Springer}\n}\n\n@article{yao2018using,\n  title={Using stacking to average Bayesian predictive distributions},\n  author={Yao, Yuling and Vehtari, Aki and Simpson, Daniel and Gelman, Andrew},\n  journal={Bayesian Analysis},\n  volume={13},\n  number={3},\n  pages={911--1007},\n  year={2018},\n  publisher={International Society for Bayesian Analysis}\n}\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjordandeklerk%2Fpyloo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjordandeklerk%2Fpyloo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjordandeklerk%2Fpyloo/lists"}