{"id":26450439,"url":"https://github.com/metriculous-ml/metriculous","last_synced_at":"2025-10-17T10:01:50.628Z","repository":{"id":54872766,"uuid":"285070291","full_name":"metriculous-ml/metriculous","owner":"metriculous-ml","description":"Measure and visualize machine learning model performance without the usual boilerplate.","archived":false,"fork":false,"pushed_at":"2024-09-13T09:31:27.000Z","size":3713,"stargazers_count":97,"open_issues_count":1,"forks_count":11,"subscribers_count":3,"default_branch":"develop","last_synced_at":"2025-03-27T02:09:43.218Z","etag":null,"topics":["classification","confusion-matrix","data-science","deep-learning","machine-learning","model-comparsion","model-evaluation","model-selection","precision-recall-curve","python","regression","residual-plot","roc-curve","statistics","visual-analysis"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metriculous-ml.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-04T18:52:03.000Z","updated_at":"2025-02-19T02:16:55.000Z","dependencies_parsed_at":"2024-09-13T21:16:08.512Z","dependency_job_id":null,"html_url":"https://github.com/metriculous-ml/metriculous","commit_stats":{"total_commits":79,"total_committers":4,"mean_commits":19.75,"dds":0.5063291139240507,"last_synced_commit":"a534ef6761406141a51d7d059a1690bff944db0f"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metriculous-ml%2Fmetriculous","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metriculous-ml%2Fmetriculous/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metriculous-ml%2Fmetriculous/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metriculous-ml%2Fmetriculous/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metriculous-ml","download_url":"https://codeload.github.com/metriculous-ml/metriculous/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248702099,"owners_count":21148114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","confusion-matrix","data-science","deep-learning","machine-learning","model-comparsion","model-evaluation","model-selection","precision-recall-curve","python","regression","residual-plot","roc-curve","statistics","visual-analysis"],"created_at":"2025-03-18T15:37:56.392Z","updated_at":"2025-10-17T10:01:50.533Z","avatar_url":"https://github.com/metriculous-ml.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://mybinder.org/v2/gh/metriculous-ml/metriculous/master?filepath=notebooks\"\u003e\n        \u003cimg \n            src=\"https://mybinder.org/badge_logo.svg\"\n            alt=\"Launch Binder\"\n        /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/metriculous-ml/metriculous/actions\"\u003e\n        \u003cimg \n            src=\"https://github.com/metriculous-ml/metriculous/workflows/CI/badge.svg?branch=master\"\n            alt=\"Current GitHub Actions build status\" \n        /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"http://mypy-lang.org/\"\u003e\n        \u003cimg\n            src=\"https://img.shields.io/badge/mypy-checked-blue\"\n            alt=\"Checked with mypy\" \n        /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://badge.fury.io/py/metriculous\"\u003e\n        \u003cimg\n            src=\"https://badge.fury.io/py/metriculous.svg\"\n            alt=\"PyPI version\"\n        /\u003e\n    \u003c/a\u003e\n    \u003cimg \n        src=\"https://img.shields.io/pypi/pyversions/metriculous\"\n        alt=\"PyPI - Python Version\"\n    \u003e\n    \u003cimg \n        src=\"https://img.shields.io/github/license/metriculous-ml/metriculous\"\n        alt=\"License MIT\"\n    \u003e\n    \u003ca href=\"https://luminovo.ai/\"\u003e\n        \u003cimg\n            src=\"https://img.shields.io/badge/friends%20with-luminovo.AI-green\"\n            alt=\"Friends with Luminovo.AI\"\n        \u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\n# __`metriculous`__\n\nMeasure, visualize, and compare machine learning model performance without the usual boilerplate.\nBreaking API improvements to be expected.\n\n\n# Installation\n```console\n$ pip install metriculous\n```\n\nOr, for the latest unreleased version:\n```console\n$ pip install git+https://github.com/metriculous-ml/metriculous.git\n```\n\n\n# Comparing Regression Models  [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/metriculous-ml/metriculous/master?filepath=notebooks%2Fquickstart_regression.py)\n\u003cdetails\u003e\u003csummary\u003eClick to see more code\u003c/summary\u003e\n\u003cp\u003e\n\n```python\nimport numpy as np\n\n# Mock the ground truth, a one-dimensional array of floats\nground_truth = np.random.random(300)\n\n# Mock the output of a few models\nperfect_model = ground_truth\nnoisy_model = ground_truth + 0.1 * np.random.randn(*ground_truth.shape)\nrandom_model = np.random.randn(*ground_truth.shape)\nzero_model = np.zeros_like(ground_truth)\n```\n\u003c/p\u003e\n\u003c/details\u003e\n\n```python\nimport metriculous\n\nmetriculous.compare_regressors(\n    ground_truth=ground_truth,\n    model_predictions=[perfect_model, noisy_model, random_model, zero_model],\n    model_names=[\"Perfect Model\", \"Noisy Model\", \"Random Model\", \"Zero Model\"],\n).save_html(\"comparison.html\").display()\n```\n\nThis will save an HTML file with common regression metrics and charts, and if you are working in a [Jupyter notebook](https://github.com/jupyter/notebook) will display the output right in front of you:\n\n\n![Screenshot of Metriculous Regression Metrics](./imgs/metriculous_regression_screen_shot_table.png)\n![Screenshot of Metriculous Regression Figures](./imgs/metriculous_regression_screen_shot_figures.png)\n\n\n# Comparing Classification Models [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/metriculous-ml/metriculous/master?filepath=notebooks%2Fquickstart_classification.py)\n\u003cdetails\u003e\u003csummary\u003eClick to see more code\u003c/summary\u003e\n\u003cp\u003e\n\n```python\nimport numpy as np\n\n\ndef normalize(array2d: np.ndarray) -\u003e np.ndarray:\n    return array2d / array2d.sum(axis=1, keepdims=True)\n\n\nclass_names = [\"Cat\", \"Dog\", \"Pig\"]\nnum_classes = len(class_names)\nnum_samples = 500\n\n# Mock ground truth\nground_truth = np.random.choice(range(num_classes), size=num_samples, p=[0.5, 0.4, 0.1])\n\n# Mock model predictions\nperfect_model = np.eye(num_classes)[ground_truth]\nnoisy_model = normalize(\n    perfect_model + 2 * np.random.random((num_samples, num_classes))\n)\nrandom_model = normalize(np.random.random((num_samples, num_classes)))\n```\n\n\u003c/p\u003e\n\u003c/details\u003e\n\n```python\nimport metriculous\n\nmetriculous.compare_classifiers(\n    ground_truth=ground_truth,\n    model_predictions=[perfect_model, noisy_model, random_model],\n    model_names=[\"Perfect Model\", \"Noisy Model\", \"Random Model\"],\n    class_names=class_names,\n    one_vs_all_figures=True,\n).save_html(\"comparison.html\").display()\n```\n\n![Screenshot of Metriculous Classification Table](./imgs/metriculous_classification_table.png)\n\n![Screenshot of Metriculous Classification Figures](./imgs/metriculous_classification_figures_1.png)\n\n![Screenshot of Metriculous Classification Figures](./imgs/metriculous_classification_figures_2.png)\n\n![Screenshot of Metriculous Classification Figures](./imgs/metriculous_classification_figures_3.png)\n\n\n# Development\n\n### Poetry\nThis project uses [poetry](https://poetry.eustace.io/) to manage\ndependencies. Please make sure it is installed for the required python version. Then install the dependencies with `poetry install`.\n\n### Makefile\nA Makefile is used to automate common development workflows. Type `make` or `make help` to see a list of available commands. Before committing changes it is recommended to run `make format check test`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetriculous-ml%2Fmetriculous","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetriculous-ml%2Fmetriculous","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetriculous-ml%2Fmetriculous/lists"}