https://github.com/benhamner/Metrics

Machine learning evaluation metrics, implemented in Python, R, Haskell, and MATLAB / Octave
https://github.com/benhamner/Metrics

Last synced: 1 day ago
JSON representation

Machine learning evaluation metrics, implemented in Python, R, Haskell, and MATLAB / Octave

• Host: GitHub
• URL: https://github.com/benhamner/Metrics
• Owner: benhamner
• License: other
• Created: 2012-06-12T15:52:50.000Z (over 12 years ago)
• Default Branch: master
• Last Pushed: 2023-01-11T06:01:24.000Z (almost 2 years ago)
• Last Synced: 2024-10-29T15:39:57.451Z (15 days ago)
• Language: Python
• Homepage:
• Size: 784 KB
• Stars: 1,628
• Watchers: 87
• Forks: 454
• Open Issues: 36
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
**Note: the current releases of this toolbox are a beta release, to test working with Haskell's, Python's, and R's code repositories.**

![Build Status](https://travis-ci.org/benhamner/Metrics.png)

**Metrics** provides implementations of various supervised machine learning evaluation metrics in the following languages:

 

 - [**Python**](https://github.com/benhamner/Metrics/tree/master/Python) ```easy_install ml_metrics```

 - [**R**](https://github.com/benhamner/Metrics/tree/master/R) ```install.packages("Metrics")``` from the R prompt

 - [**Haskell**](https://github.com/benhamner/Metrics/tree/master/Haskell) ```cabal install Metrics```

 - [**MATLAB / Octave**](https://github.com/benhamner/Metrics/tree/master/MATLAB) (clone the repo & run setup from the MATLAB command line)

For more detailed installation instructions, see the README for each implementation.

EVALUATION METRICS

------------------

Evaluation MetricPythonRHaskellMATLAB / Octave

Absolute Error (AE)✓✓✓✓

Average Precision at K (APK, AP@K)✓✓✓✓

Area Under the ROC (AUC)✓✓✓✓

Classification Error (CE)✓✓✓✓

F1 Score (F1) ✓ 

Gini   ✓

Levenshtein✓ ✓✓

Log Loss (LL)✓✓✓✓

Mean Log Loss (LogLoss)✓✓✓✓

Mean Absolute Error (MAE)✓✓✓✓

Mean Average Precision at K (MAPK, MAP@K)✓✓✓✓

Mean Quadratic Weighted Kappa✓✓ ✓

Mean Squared Error (MSE)✓✓✓✓

Mean Squared Log Error (MSLE)✓✓✓✓

Normalized Gini   ✓

Quadratic Weighted Kappa✓✓ ✓

Relative Absolute Error (RAE) ✓  

Root Mean Squared Error (RMSE)✓✓✓✓

Relative Squared Error (RSE) ✓  

Root Relative Squared Error (RRSE) ✓  

Root Mean Squared Log Error (RMSLE)✓✓✓✓

Squared Error (SE)✓✓✓✓

Squared Log Error (SLE)✓✓✓✓

TO IMPLEMENT

------------

 - F1 score

 - Multiclass log loss

 - Lift

 - Average Precision for binary classification

 - precision / recall break-even point

 - cross-entropy

 - True Pos / False Pos / True Neg / False Neg rates

 - precision / recall / sensitivity / specificity

 - mutual information

HIGHER LEVEL TRANSFORMATIONS TO HANDLE

--------------------------------------

 - GroupBy / Reduce

 - Weight individual samples or groups

PROPERTIES METRICS CAN HAVE

---------------------------

(Nonexhaustive and to be added in the future)

 - Min or Max (optimize through minimization or maximization)

 - Binary Classification

   - Scores predicted class labels

   - Scores predicted ranking (most likely to least likely for being in one class)

   - Scores predicted probabilities

 - Multiclass Classification

   - Scores predicted class labels

   - Scores predicted probabilities

 - Regression

 - Discrete Rater Comparison (confusion matrix)