https://github.com/metriculous-ml/metriculous

Measure and visualize machine learning model performance without the usual boilerplate.
https://github.com/metriculous-ml/metriculous

classification confusion-matrix data-science deep-learning machine-learning model-comparsion model-evaluation model-selection precision-recall-curve python regression residual-plot roc-curve statistics visual-analysis

Last synced: 21 days ago
JSON representation

Measure and visualize machine learning model performance without the usual boilerplate.

• Host: GitHub
• URL: https://github.com/metriculous-ml/metriculous
• Owner: metriculous-ml
• License: mit
• Created: 2020-08-04T18:52:03.000Z (over 4 years ago)
• Default Branch: develop
• Last Pushed: 2024-09-13T09:31:27.000Z (8 months ago)
• Last Synced: 2025-03-27T02:09:43.218Z (about 1 month ago)
• Topics: classification, confusion-matrix, data-science, deep-learning, machine-learning, model-comparsion, model-evaluation, model-selection, precision-recall-curve, python, regression, residual-plot, roc-curve, statistics, visual-analysis
• Language: Python
• Homepage:
• Size: 3.54 MB
• Stars: 97
• Watchers: 3
• Forks: 11
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

        


    

        

    

    

        

    

    

        

    

    

        

    

    

    

    

        

    



# __`metriculous`__

Measure, visualize, and compare machine learning model performance without the usual boilerplate.

Breaking API improvements to be expected.

# Installation

```console

$ pip install metriculous

```

Or, for the latest unreleased version:

```console

$ pip install git+https://github.com/metriculous-ml/metriculous.git

```

# Comparing Regression Models  [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/metriculous-ml/metriculous/master?filepath=notebooks%2Fquickstart_regression.py)

Click to see more code



```python

import numpy as np

# Mock the ground truth, a one-dimensional array of floats

ground_truth = np.random.random(300)

# Mock the output of a few models

perfect_model = ground_truth

noisy_model = ground_truth + 0.1 * np.random.randn(*ground_truth.shape)

random_model = np.random.randn(*ground_truth.shape)

zero_model = np.zeros_like(ground_truth)

```



```python

import metriculous

metriculous.compare_regressors(

    ground_truth=ground_truth,

    model_predictions=[perfect_model, noisy_model, random_model, zero_model],

    model_names=["Perfect Model", "Noisy Model", "Random Model", "Zero Model"],

).save_html("comparison.html").display()

```

This will save an HTML file with common regression metrics and charts, and if you are working in a [Jupyter notebook](https://github.com/jupyter/notebook) will display the output right in front of you:

![Screenshot of Metriculous Regression Metrics](./imgs/metriculous_regression_screen_shot_table.png)

![Screenshot of Metriculous Regression Figures](./imgs/metriculous_regression_screen_shot_figures.png)

# Comparing Classification Models [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/metriculous-ml/metriculous/master?filepath=notebooks%2Fquickstart_classification.py)

Click to see more code



```python

import numpy as np

def normalize(array2d: np.ndarray) -> np.ndarray:

    return array2d / array2d.sum(axis=1, keepdims=True)

class_names = ["Cat", "Dog", "Pig"]

num_classes = len(class_names)

num_samples = 500

# Mock ground truth

ground_truth = np.random.choice(range(num_classes), size=num_samples, p=[0.5, 0.4, 0.1])

# Mock model predictions

perfect_model = np.eye(num_classes)[ground_truth]

noisy_model = normalize(

    perfect_model + 2 * np.random.random((num_samples, num_classes))

)

random_model = normalize(np.random.random((num_samples, num_classes)))

```



```python

import metriculous

metriculous.compare_classifiers(

    ground_truth=ground_truth,

    model_predictions=[perfect_model, noisy_model, random_model],

    model_names=["Perfect Model", "Noisy Model", "Random Model"],

    class_names=class_names,

    one_vs_all_figures=True,

).save_html("comparison.html").display()

```

![Screenshot of Metriculous Classification Table](./imgs/metriculous_classification_table.png)

![Screenshot of Metriculous Classification Figures](./imgs/metriculous_classification_figures_1.png)

![Screenshot of Metriculous Classification Figures](./imgs/metriculous_classification_figures_2.png)

![Screenshot of Metriculous Classification Figures](./imgs/metriculous_classification_figures_3.png)

# Development

### Poetry

This project uses [poetry](https://poetry.eustace.io/) to manage

dependencies. Please make sure it is installed for the required python version. Then install the dependencies with `poetry install`.

### Makefile

A Makefile is used to automate common development workflows. Type `make` or `make help` to see a list of available commands. Before committing changes it is recommended to run `make format check test`.