{"id":13704458,"url":"https://github.com/ModelOriented/auditor","last_synced_at":"2025-05-05T09:34:01.035Z","repository":{"id":34505470,"uuid":"110183650","full_name":"ModelOriented/auditor","owner":"ModelOriented","description":"Model verification, validation, and error analysis","archived":false,"fork":false,"pushed_at":"2024-01-09T23:32:23.000Z","size":53090,"stargazers_count":58,"open_issues_count":9,"forks_count":14,"subscribers_count":20,"default_branch":"master","last_synced_at":"2024-10-28T17:24:21.779Z","etag":null,"topics":["classification","error-analysis","explainable-artificial-intelligence","machine-learning","model-validation","regression-models","residuals","xai"],"latest_commit_sha":null,"homepage":"https://modeloriented.github.io/auditor/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ModelOriented.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":"auditor.Rproj","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-10T00:51:00.000Z","updated_at":"2024-03-12T12:42:32.000Z","dependencies_parsed_at":"2022-08-08T01:01:21.609Z","dependency_job_id":null,"html_url":"https://github.com/ModelOriented/auditor","commit_stats":{"total_commits":473,"total_committers":13,"mean_commits":36.38461538461539,"dds":"0.41014799154334036","last_synced_commit":"1f2fdc125017e936b7936657e7fc4305bc2d4d06"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2Fauditor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2Fauditor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2Fauditor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2Fauditor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ModelOriented","download_url":"https://codeload.github.com/ModelOriented/auditor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224439981,"owners_count":17311563,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","error-analysis","explainable-artificial-intelligence","machine-learning","model-validation","regression-models","residuals","xai"],"created_at":"2024-08-02T21:01:10.169Z","updated_at":"2024-11-13T11:31:25.617Z","avatar_url":"https://github.com/ModelOriented.png","language":"R","readme":"# Model verification, validation, and error analysis \u003cimg src=\"man/figures/logo.png\" align=\"right\" width=\"150\"/\u003e\n\n[![CRAN\\_Status\\_Badge](http://www.r-pkg.org/badges/version/auditor)](https://cran.r-project.org/package=auditor)\n[![R build status](https://github.com/ModelOriented/auditor/workflows/R-CMD-check/badge.svg)](https://github.com/ModelOriented/auditor/actions)\n[![Coverage Status](https://codecov.io/gh/ModelOriented/auditor/branch/master/graph/badge.svg)](https://codecov.io/gh/ModelOriented/auditor?branch=master)\n[![](https://cranlogs.r-pkg.org/badges/grand-total/auditor)](https://cranlogs.r-pkg.org/badges/grand-total/auditor)\n[![DrWhy-eXtrAI](https://img.shields.io/badge/DrWhy-eXtrAI-4378bf)](http://drwhy.ai/#eXtraAI)\n\n## Overview\n\nPackage `auditor` is a tool for model-agnostic validation. Implemented\ntechniques facilitate assessing and comparing the goodness of fit and\nperformance of models. In addition, they may be used for the analysis of\nthe similarity of residuals and for the identification of outliers and\ninfluential observations. The examination is carried out by diagnostic\nscores and visual verification. Due to the flexible and consistent\ngrammar, it is simple to validate models of any classes.\n\n[An up-to-date paper about auditor](https://arxiv.org/abs/1809.07763) and [a shorter version in The R-Journal](https://journal.r-project.org/).\n\n`auditor` is a part of [DrWhy](drwhy.ai) collection of tools for Visual\nExploration, Explanation and Debugging of Predictive Models.\n\nauditor’s pipeline: *model %\\\u003e% DALEX::explain() %\\\u003e% plot(type=…)*\n\n## Installation\n\nStable version from CRAN:\n\n``` r\ninstall.packages(\"auditor\")\n```\n\nDeveloper version from GitHub:\n\n``` r\nsource(\"https://install-github.me/ModelOriented/auditor\")\n\n# or with the devtools package\ndevtools::install_github(\"ModelOriented/auditor\")\n```\n\n## Demo\n\nRun the code below or try the auditor.\n\n``` r\nlibrary(auditor)\nlibrary(randomForest)\ndata(mtcars)\n\n# fitting models\nmodel_lm \u003c- lm(mpg ~ ., data = mtcars)\nset.seed(123)\nmodel_rf \u003c- randomForest(mpg ~ ., data = mtcars)\n\n# creating objects with 'explain' function from the package DALEX\n# that contains all necessary components required for further processing\nexp_lm \u003c- DALEX::explain(model_lm, data = mtcars, y = mtcars$mpg,  verbose = FALSE)\nexp_rf \u003c- DALEX::explain(model_rf, data = mtcars, y = mtcars$mpg, label = \"rf\", verbose = FALSE)\n\n# create explanation  objects\nmr_lm \u003c- model_residual(exp_lm)\nmr_rf \u003c- model_residual(exp_rf)\n\n# generating plots\nplot_residual(mr_lm, mr_rf, variable = \"wt\", smooth = TRUE)\n```\n\n![](man/figures/demo.png)\u003c!-- --\u003e\n\n## More Resources\n\n  - [News](NEWS.md)\n\n  - The [website about\n    auditor](https://modeloriented.github.io/auditor/)\n\n  - A [preprint of the article about\n    auditor](https://arxiv.org/abs/1809.07763) is available on arxiv.\n    \n  - A [Cheatsheet](https://github.com/ModelOriented/auditor/blob/master/materials/auditor_cheatsheet.pdf)\n\n\n## Short overview of plots\n\nColumn type contains character that should be passed to parameter\n`type=` when using `plot()` function. `Regr` and `Class` columns\nindicate whether plot can be used for regression and classification\nmodels.\n\n| Name of a plot                                      | Function                                                                                                  | Interactive version                                                                                        | Type                | Regr | Class |\n| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------- | ---- | ----- |\n| Autocorrelation Function                            | [plot\\_acf()](https://modeloriented.github.io/auditor/reference/plot_acf.html)                            | [plotD3\\_acf()](https://modeloriented.github.io/auditor/reference/plotD3_acf.html)                         | “acf”               | yes  | yes   |\n| Autocorrelation                                     | [plot\\_autocorrelation()](https://modeloriented.github.io/auditor/reference/plot_autocorrelation.html)    | [plotD3\\_autocorrelation()](https://modeloriented.github.io/auditor/reference/plotD3_autocorrelation.html) | “autocorrelation”   | yes  | yes   |\n| Influence of Observations                           | [plot\\_cooksdistance()](https://modeloriented.github.io/auditor/reference/plot_cooksdistance.html)        | [plotD3\\_cooksdistance()](https://modeloriented.github.io/auditor/reference/plotD3_cooksdistance.html)     | “cooksdistance”     | yes  | yes   |\n| Half-Normal                                         | [plot\\_halfnormal()](https://modeloriented.github.io/auditor/reference/plot_halfnormal.html)              | [plotD3\\_halfnormal()](https://modeloriented.github.io/auditor/reference/plotD3_halfnormal.html)           | “halfnormal”        | yes  | yes   |\n| LIFT Chart                                          | [plot\\_lift()](https://modeloriented.github.io/auditor/reference/plot_lift.html)                          | [plotD3\\_lift()](https://modeloriented.github.io/auditor/reference/plotD3_lift.html)                       | “lift”              | no   | yes   |\n| Model Correlation                                   | [plot\\_correlation()](https://modeloriented.github.io/auditor/reference/plot_correlation.html)            | \\-                                                                                                         | “correlation”       | yes  | yes   |\n| Principal Component Analysis of Models              | [plot\\_pca()](https://modeloriented.github.io/auditor/reference/plot_pca.html)                            | \\-                                                                                                         | “pca”               | yes  | yes   |\n| Model Ranking Radar Plot                            | [plot\\_radar()](https://modeloriented.github.io/auditor/reference/plot_radar.html)                        | \\-                                                                                                         | “radar”             | yes  | yes   |\n| Predicted Response vs Actual or Variable Values     | [plot\\_prediction()](https://modeloriented.github.io/auditor/reference/plot_prediction.html)              | [plotD3\\_prediction()](https://modeloriented.github.io/auditor/reference/plotD3_prediction.html)           | “prediction”        | yes  | yes   |\n| Regression Error Characteristic Curve (REC)         | [plot\\_rec()](https://modeloriented.github.io/auditor/reference/plot_rec.html)                            | [plotD3\\_rec()](https://modeloriented.github.io/auditor/reference/plotD3_rec.html)                         | “rec”               | yes  | yes   |\n| Plot Residuals vs Actual, Fitted or Variable Values | [plot\\_residual()](https://modeloriented.github.io/auditor/reference/plot_residual.html)                  | [plotD3\\_residual()](https://modeloriented.github.io/auditor/reference/plotD3_residual.html)               | “residual”          | yes  | yes   |\n| Residual Boxplot                                    | [plot\\_residual\\_boxplot()](https://modeloriented.github.io/auditor/reference/plot_residual_boxplot.html) | \\-                                                                                                         | “residual\\_boxplot” | yes  | yes   |\n| Residual Density                                    | [plot\\_residual\\_density()](https://modeloriented.github.io/auditor/reference/plot_residual_density.html) | \\-                                                                                                         | “residual\\_density” | yes  | yes   |\n| Receiver Operating Characteristic (ROC) Curve       | [plot\\_roc()](https://modeloriented.github.io/auditor/reference/plot_roc.html)                            | [plotD3_roc](https://modeloriented.github.io/auditor/reference/plotD3_roc.html)                                                                                                         | “roc”               | no   | yes   |\n| Regression Receiver Operating Characteristic (RROC) | [plot\\_rroc()](https://modeloriented.github.io/auditor/reference/plot_rroc.html)                          | [plotD3_rroc](https://modeloriented.github.io/auditor/reference/plotD3_rroc.html)                                                                                                         | “rroc”              | yes  | yes   |\n| Scale-Location Plot                                 | [plot\\_scalelocation()](https://modeloriented.github.io/auditor/reference/plot_scalelocation.html)        | [plotD3\\_scalelocation()](https://modeloriented.github.io/auditor/reference/plotD3_scalelocation.html)     | “scalelocation”     | yes  | yes   |\n| Two-sided Cumulative Distribution Function          | [plot\\_tsecdf()](https://modeloriented.github.io/auditor/reference/plot_tsecdf.html)                      | \\-                                                                                                         | “tsecdf”            | yes  | yes   |\n\n\n##  Acknowledgments\nWork on this package was financially supported by\n    the `NCN Opus grant 2016/21/B/ST6/02176`.\n","funding_links":[],"categories":["Tools"],"sub_categories":["Performance (\u0026 Automated ML)"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FModelOriented%2Fauditor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FModelOriented%2Fauditor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FModelOriented%2Fauditor/lists"}