https://github.com/cdpierse/pyinfer

Pyinfer is a model agnostic tool for ML developers and researchers to benchmark the inference statistics for machine learning models or functions.
https://github.com/cdpierse/pyinfer

developer-tools inference inference-stats machine-learning python

Last synced: 11 months ago
JSON representation

Pyinfer is a model agnostic tool for ML developers and researchers to benchmark the inference statistics for machine learning models or functions.

• Host: GitHub
• URL: https://github.com/cdpierse/pyinfer
• Owner: cdpierse
• License: apache-2.0
• Created: 2020-10-25T18:04:42.000Z (over 5 years ago)
• Default Branch: main
• Last Pushed: 2021-02-19T15:11:58.000Z (over 5 years ago)
• Last Synced: 2025-04-14T14:22:05.759Z (about 1 year ago)
• Topics: developer-tools, inference, inference-stats, machine-learning, python
• Language: Python
• Homepage: https://pyinfer.readthedocs.io
• Size: 498 KB
• Stars: 24
• Watchers: 3
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          


    

        

    



[![build](https://circleci.com/gh/cdpierse/pyinfer.svg?style=shield&circle-token=2c8717a6df1a49242ab0f694f8ab4bb58d376c91)](https://app.circleci.com/pipelines/github/cdpierse/pyinfer)



[![docs](https://readthedocs.org/projects/databay/badge/?version=latest&style=shield&circle)](https://pyinfer.readthedocs.io)

Pyinfer is a model agnostic tool for ML developers and researchers to benchmark the inference statistics for machine learning models or functions.

## Installation

```python

pip install pyinfer

```

## Overview

### Inference Report

`InferenceReport` is for reporting inference statistics on a single model artifact. To create a valid report simply pass it a callable model function or method, valid input(s), and either **n_iterations** or **n_seconds** to determine what interval the report uses for its run duration. Check out the docs for more information on the optional parameters that can be passed. 



    

        

    



### Multi Inference Report

`MultiInferenceReport` is for reporting inference statistics on a list of model artifacts. To create a valid multi report pass it a list of callable model functions or methods, a list of valid input(s), and either **n_iterations** or **n_seconds** to determine what interval the report uses for its run duration. Check out the docs for more information on the optional parameters that can be passed.



    

        

    



### Example Outputs

**Table Report**



    

        

    



**Run Plot**



    

        

    



### Stats Currently Included

- Success Rate - Number of successful inferences within a specified time range.

- Failures - Number of inferences above specified time range.

- Time Taken - Total time taken to run all inferences.

- Inference Per Second - Estimate of how many inferences per second the selected model can perform.

- Max Run - The max time taken to perform an inference for a given run.

- Min Run - The min time taken to perform an inference for a given run.

- Std - The Standard deviation between runs.

- Mean - The mean run time.

- Median - The median run time.

- IQR - The inter quartile range of the runs.

- Cores Logical - The number of logical cores on the host machine.

- Cores Physical - The number of physical Cores on the host machine.

### Planned Future Stats

- Model Size - Information relating to the size of the model in bytes. 

- GPU Stat Support - Information about if GPU is available and if it is being utilized.