{"id":15655094,"url":"https://github.com/lorentzenchr/model-diagnostics","last_synced_at":"2025-04-09T18:21:18.735Z","repository":{"id":65846019,"uuid":"534117717","full_name":"lorentzenchr/model-diagnostics","owner":"lorentzenchr","description":"Tools for diagnostics and assessment of (machine learning) models","archived":false,"fork":false,"pushed_at":"2025-03-01T15:05:53.000Z","size":6610,"stargazers_count":34,"open_issues_count":12,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-02T13:50:23.809Z","etag":null,"topics":["bias-detection","calibration","machine-learning","performance-metrics","python"],"latest_commit_sha":null,"homepage":"https://lorentzenchr.github.io/model-diagnostics/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lorentzenchr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-08T08:22:56.000Z","updated_at":"2025-03-01T15:01:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"57f96c4c-9cf3-4426-a26c-fcf1fac94838","html_url":"https://github.com/lorentzenchr/model-diagnostics","commit_stats":{"total_commits":209,"total_committers":3,"mean_commits":69.66666666666667,"dds":"0.014354066985645897","last_synced_commit":"54dac93d0f858c03668dd8ec370357b66597d6a0"},"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lorentzenchr%2Fmodel-diagnostics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lorentzenchr%2Fmodel-diagnostics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lorentzenchr%2Fmodel-diagnostics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lorentzenchr%2Fmodel-diagnostics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lorentzenchr","download_url":"https://codeload.github.com/lorentzenchr/model-diagnostics/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248085481,"owners_count":21045168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bias-detection","calibration","machine-learning","performance-metrics","python"],"created_at":"2024-10-03T12:56:04.054Z","updated_at":"2025-04-09T18:21:18.717Z","avatar_url":"https://github.com/lorentzenchr.png","language":"Python","funding_links":[],"categories":["Tools"],"sub_categories":["Performance (\u0026 Automated ML)"],"readme":"# model-diagnostics\n\n| | |\n| --- | --- |\n| CI/CD |[![CI - Test](https://github.com/lorentzenchr/model-diagnostics/actions/workflows/test.yml/badge.svg)](https://github.com/lorentzenchr/model-diagnostics/actions/workflows/test.yml) [![Coverage](https://codecov.io/github/lorentzenchr/model-diagnostics/coverage.svg?branch=main)](https://codecov.io/gh/lorentzenchr/model-diagnostics)\n| Docs | [![Docs](https://github.com/lorentzenchr/model-diagnostics/actions/workflows/docs.yml/badge.svg)](https://github.com/lorentzenchr/model-diagnostics/actions/workflows/docs.yml)\n| Package | [![PyPI - Version](https://img.shields.io/pypi/v/model-diagnostics.svg?logo=pypi\u0026label=PyPI\u0026logoColor=gold)](https://pypi.org/project/model-diagnostics/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/model-diagnostics.svg?color=blue\u0026label=Downloads\u0026logo=pypi\u0026logoColor=gold)](https://pypi.org/project/model-diagnostics/) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/model-diagnostics.svg?logo=python\u0026label=Python\u0026logoColor=gold)](https://pypi.org/project/model-diagnostics/) |\n| Meta | [![Hatch project](https://img.shields.io/badge/%F0%9F%A5%9A-Hatch-4051b5.svg)](https://github.com/pypa/hatch) [![linting - Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![types - Mypy](https://img.shields.io/badge/types-Mypy-blue.svg)](https://github.com/python/mypy) [![License - MIT](https://img.shields.io/badge/license-MIT-9400d3.svg)](https://spdx.org/licenses/)\n\n**Tools for diagnostics and assessment of (machine learning) models**\n\nHighlights:\n\n- All common point predictions covered: mean, median, quantiles, expectiles.\n- Assess model calibration with [identification functions](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/calibration/identification/#model_diagnostics.calibration.identification.identification_function) (generalized residuals), [compute_bias](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/calibration/identification/#model_diagnostics.calibration.identification.compute_bias) and [compute_marginal](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/calibration/identification/#model_diagnostics.calibration.identification.compute_marginal).\n- Assess calibration and bias graphically\n    - [reliability diagrams](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/calibration/plots/#model_diagnostics.calibration.plots.plot_reliability_diagram) for auto-calibration\n    - [bias plots](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/calibration/plots/#model_diagnostics.calibration.plots.plot_bias) for conditional calibration\n    - [marginal plots](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/calibration/plots/#model_diagnostics.calibration.plots.plot_marginal) for average `y_obs`, `y_pred` and partial dependence for one feature\n- Assess the predictive performance of models\n    - strictly consistent, homogeneous [scoring functions](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/scoring/scoring/)\n    - [score decomposition](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/scoring/scoring/#model_diagnostics.scoring.scoring.decompose) into miscalibration, discrimination and uncertainty\n- Choose your plot backend, either [matplotlib](https://matplotlib.org) or [plotly](https://plotly.com/python/), e.g., via [set_config](https://lorentzenchr.github.io/model-diagnostics/reference/model_diagnostics/#model_diagnostics.set_config).\n\n:rocket: To our knowledge, this is the first python package to offer reliability diagrams for quantiles and expectiles and a score decomposition, both made available by an internal implementation of isotonic quantile/expectile regression. :rocket:\n\nRead more in the [documentation](https://lorentzenchr.github.io/model-diagnostics/).\n\nThis package relies on the giant shoulders of, among others, [polars](https://pola.rs/), [matplotlib](https://matplotlib.org), [scipy](https://scipy.org) and [scikit-learn](https://scikit-learn.org).\n\n**Installation**\n\n```\npip install model-diagnostics\n```\n\n**Contributions**\n\nContributions are warmly welcome!\nWhen contributing, you agree that your contributions will be subject to the [MIT License](https://github.com/lorentzenchr/model-diagnostics/blob/main/LICENSE).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Florentzenchr%2Fmodel-diagnostics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Florentzenchr%2Fmodel-diagnostics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Florentzenchr%2Fmodel-diagnostics/lists"}