{"id":13780357,"url":"https://github.com/ModelOriented/modelStudio","last_synced_at":"2025-05-11T13:32:11.528Z","repository":{"id":53592350,"uuid":"196159953","full_name":"ModelOriented/modelStudio","owner":"ModelOriented","description":"📍 Interactive Studio for Explanatory Model Analysis","archived":false,"fork":false,"pushed_at":"2023-08-31T11:00:20.000Z","size":37994,"stargazers_count":325,"open_issues_count":4,"forks_count":32,"subscribers_count":21,"default_branch":"master","last_synced_at":"2024-10-30T06:54:57.270Z","etag":null,"topics":["ai","explainable","explainable-ai","explainable-machine-learning","explanatory-model-analysis","human","iml","interactive","interactivity","interpretability","interpretable","interpretable-machine-learning","learning","machine","model","model-visualization","r","visualization","xai"],"latest_commit_sha":null,"homepage":"https://doi.org/10.1007/s10618-023-00924-w","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ModelOriented.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":".github/CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null},"funding":{"github":"hbaniecki"}},"created_at":"2019-07-10T07:59:07.000Z","updated_at":"2024-10-25T11:46:16.000Z","dependencies_parsed_at":"2022-08-13T02:41:03.208Z","dependency_job_id":"8ca32605-6a23-4d7c-b677-5782c137192a","html_url":"https://github.com/ModelOriented/modelStudio","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2FmodelStudio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2FmodelStudio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2FmodelStudio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ModelOriented%2FmodelStudio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ModelOriented","download_url":"https://codeload.github.com/ModelOriented/modelStudio/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225056808,"owners_count":17414214,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","explainable","explainable-ai","explainable-machine-learning","explanatory-model-analysis","human","iml","interactive","interactivity","interpretability","interpretable","interpretable-machine-learning","learning","machine","model","model-visualization","r","visualization","xai"],"created_at":"2024-08-03T18:01:14.875Z","updated_at":"2024-11-17T15:31:08.937Z","avatar_url":"https://github.com/ModelOriented.png","language":"R","readme":"# Interactive Studio for Explanatory Model Analysis\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"man/figures/logo.gif\" width=\"200\"/\u003e\n\u003c/p\u003e\n\n[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/modelStudio)](https://cran.r-project.org/package=modelStudio)\n[![R build status](https://github.com/ModelOriented/modelStudio/workflows/R-CMD-check/badge.svg)](https://github.com/ModelOriented/modelStudio/actions?query=workflow%3AR-CMD-check)\n[![Codecov test coverage](https://codecov.io/gh/ModelOriented/modelStudio/branch/master/graph/badge.svg)](https://app.codecov.io/gh/ModelOriented/modelStudio?branch=master)\n[![](https://cranlogs.r-pkg.org/badges/grand-total/modelStudio)](https://cranlogs.r-pkg.org/badges/grand-total/modelStudio)\n[![JOSS-status](https://joss.theoj.org/papers/9eec8c9d1969fbd44b3ea438a74af911/status.svg)](https://joss.theoj.org/papers/9eec8c9d1969fbd44b3ea438a74af911)\n\n## Overview\n\nThe `modelStudio` package **automates the explanatory analysis of machine learning predictive models**. It generates advanced interactive model explanations in the form of a **serverless HTML site** with only one line of code. This tool is model-agnostic, therefore compatible with most of the black-box predictive models and frameworks (e.g.\u0026nbsp;`mlr/mlr3`, `xgboost`, `caret`, `h2o`, `parsnip`, `tidymodels`, `scikit-learn`, `lightgbm`, `keras/tensorflow`).\n\nThe main `modelStudio()` function computes various (instance and model-level) explanations and produces a\u0026nbsp;**customisable dashboard**, which consists of multiple panels for plots with their short descriptions. It is possible to easily **save** the dashboard and\u0026nbsp;**share** it with others. Tools for [Explanatory Model Analysis](https://ema.drwhy.ai) unite with tools for Exploratory Data Analysis to give a broad overview of the model behavior.\n\n\u003cp align=\"center\"\u003e\u003cb\u003e\n\u003ca href=\"https://rai-covid.drwhy.ai\"\u003eexplain COVID-19\u003ca\u003e \u0026emsp;\n\u003ca href=\"http://modelstudio.drwhy.ai/articles/ms-r-python-examples.html\"\u003eR \u0026 Python examples\u003ca\u003e \u0026emsp;\n\u003ca href=\"http://modelstudio.drwhy.ai/#more-resources\"\u003eMore resources\u003ca\u003e \u0026emsp;\n\u003ca href=\"https://iema.drwhy.ai\"\u003eInteractive EMA\u003ca\u003e \u0026emsp;\n\u003c/b\u003e\u003c/p\u003e\n\n[![](man/figures/demo_small.gif)](https://modelstudio.drwhy.ai/demo.html)\n\nThe `modelStudio` package is a part of the [**DrWhy.AI**](http://drwhy.ai) universe.\n\n## Installation\n\n```r\n# Install from CRAN:\ninstall.packages(\"modelStudio\")\n\n# Install the development version from GitHub:\ndevtools::install_github(\"ModelOriented/modelStudio\")\n```\n\n## Simple demo\n\n```r\nlibrary(\"DALEX\")\nlibrary(\"ranger\")\nlibrary(\"modelStudio\")\n\n# fit a model\nmodel \u003c- ranger(score ~., data = happiness_train)\n\n# create an explainer for the model    \nexplainer \u003c- explain(model,\n                     data = happiness_test,\n                     y = happiness_test$score,\n                     label = \"Random Forest\")\n\n# make a studio for the model\nmodelStudio(explainer)\n```\n\n[Save the output](https://modelstudio.drwhy.ai/#save--share) in the form of a HTML file - [**Demo Dashboard**](https://modelstudio.drwhy.ai/demo.html).\n\n[![](man/figures/demo_big.gif)](https://modelstudio.drwhy.ai/demo.html)\n\n## R \u0026 Python examples [more](https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html)\n\n-------------------------------\n\nThe `modelStudio()` function uses `DALEX` explainers created with `DALEX::explain()` or `DALEXtra::explain_*()`.\n\n```r\n# packages for the explainer objects\ninstall.packages(\"DALEX\")\ninstall.packages(\"DALEXtra\")\n```\n\n### mlr [dashboard](https://modelstudio.drwhy.ai/mlr.html)\n\nMake a studio for the regression `ranger` model on the `apartments` data.\n\n\u003cdetails open\u003e\n\u003csummary\u003e\u003cstrong\u003e\u003cem\u003ecode\u003c/em\u003e\u003c/strong\u003e\u003c/summary\u003e\n\n```r\n# load packages and data\nlibrary(mlr)\nlibrary(DALEXtra)\nlibrary(modelStudio)\n\ndata \u003c- DALEX::apartments\n\n# split the data\nindex \u003c- sample(1:nrow(data), 0.7*nrow(data))\ntrain \u003c- data[index,]\ntest \u003c- data[-index,]\n\n# fit a model\ntask \u003c- makeRegrTask(id = \"apartments\", data = train, target = \"m2.price\")\nlearner \u003c- makeLearner(\"regr.ranger\", predict.type = \"response\")\nmodel \u003c- train(learner, task)\n\n# create an explainer for the model\nexplainer \u003c- explain_mlr(model,\n                         data = test,\n                         y = test$m2.price,\n                         label = \"mlr\")\n\n# pick observations\nnew_observation \u003c- test[1:2,]\nrownames(new_observation) \u003c- c(\"id1\", \"id2\")\n\n# make a studio for the model\nmodelStudio(explainer, new_observation)\n```\n\n\u003c/details\u003e\n\n### xgboost [dashboard](https://modelstudio.drwhy.ai/xgboost.html)\n\nMake a studio for the classification `xgboost` model on the `titanic` data.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003e\u003cem\u003ecode\u003c/em\u003e\u003c/strong\u003e\u003c/summary\u003e\n\n```r\n# load packages and data\nlibrary(xgboost)\nlibrary(DALEX)\nlibrary(modelStudio)\n\ndata \u003c- DALEX::titanic_imputed\n\n# split the data\nindex \u003c- sample(1:nrow(data), 0.7*nrow(data))\ntrain \u003c- data[index,]\ntest \u003c- data[-index,]\n\ntrain_matrix \u003c- model.matrix(survived ~.-1, train)\ntest_matrix \u003c- model.matrix(survived ~.-1, test)\n\n# fit a model\nxgb_matrix \u003c- xgb.DMatrix(train_matrix, label = train$survived)\nparams \u003c- list(max_depth = 3, objective = \"binary:logistic\", eval_metric = \"auc\")\nmodel \u003c- xgb.train(params, xgb_matrix, nrounds = 500)\n\n# create an explainer for the model\nexplainer \u003c- explain(model,\n                     data = test_matrix,\n                     y = test$survived,\n                     type = \"classification\",\n                     label = \"xgboost\")\n\n# pick observations\nnew_observation \u003c- test_matrix[1:2, , drop=FALSE]\nrownames(new_observation) \u003c- c(\"id1\", \"id2\")\n\n# make a studio for the model\nmodelStudio(explainer, new_observation)\n```\n\n\u003c/details\u003e\n\n-------------------------\n\nThe `modelStudio()` function uses `dalex` explainers created with `dalex.Explainer()`.\n\n```console\n:: package for the Explainer object\npip install dalex -U\n```\n\nUse `pickle` Python module and `reticulate` R package to easily make a studio for a model.\n\n```r\n# package for pickle load\ninstall.packages(\"reticulate\")\n```\n\n### scikit-learn [dashboard](https://modelstudio.drwhy.ai/scikitlearn.html)\n\nMake a studio for the regression `Pipeline SVR` model on the `fifa` data.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003e\u003cem\u003ecode\u003c/em\u003e\u003c/strong\u003e\u003c/summary\u003e\n\nFirst, use `dalex` in Python:\n\n```python\n# load packages and data\nimport dalex as dx\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.svm import SVR\nfrom numpy import log\n\ndata = dx.datasets.load_fifa()\nX = data.drop(columns=['overall', 'potential', 'value_eur', 'wage_eur', 'nationality'], axis=1)\ny = log(data.value_eur)\n\n# split the data\nX_train, X_test, y_train, y_test = train_test_split(X, y)\n\n# fit a pipeline model\nmodel = Pipeline([('scale', StandardScaler()), ('svm', SVR())])\nmodel.fit(X_train, y_train)\n\n# create an explainer for the model\nexplainer = dx.Explainer(model, data=X_test, y=y_test, label='scikit-learn')\n\n# pack the explainer into a pickle file\nexplainer.dump(open('explainer_scikitlearn.pickle', 'wb'))\n```\n\nThen, use `modelStudio` in R:\n\n```r\n# load the explainer from the pickle file\nlibrary(reticulate)\nexplainer \u003c- py_load_object(\"explainer_scikitlearn.pickle\", pickle = \"pickle\")\n\n# make a studio for the model\nlibrary(modelStudio)\nmodelStudio(explainer, B = 5)\n```\n\n\u003c/details\u003e\n\n### lightgbm [dashboard](https://modelstudio.drwhy.ai/lightgbm.html)\n\nMake a studio for the classification `Pipeline LGBMClassifier` model on the `titanic` data.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003e\u003cem\u003ecode\u003c/em\u003e\u003c/strong\u003e\u003c/summary\u003e\n\nFirst, use `dalex` in Python:\n\n```python\n# load packages and data\nimport dalex as dx\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import StandardScaler, OneHotEncoder\nfrom sklearn.impute import SimpleImputer\nfrom sklearn.compose import ColumnTransformer\nfrom lightgbm import LGBMClassifier\n\ndata = dx.datasets.load_titanic()\nX = data.drop(columns='survived')\ny = data.survived\n\n# split the data\nX_train, X_test, y_train, y_test = train_test_split(X, y)\n\n# fit a pipeline model\nnumerical_features = ['age', 'fare', 'sibsp', 'parch']\nnumerical_transformer = Pipeline(\n  steps=[\n    ('imputer', SimpleImputer(strategy='median')),\n    ('scaler', StandardScaler())\n  ]\n)\ncategorical_features = ['gender', 'class', 'embarked']\ncategorical_transformer = Pipeline(\n  steps=[\n    ('imputer', SimpleImputer(strategy='constant', fill_value='missing')),\n    ('onehot', OneHotEncoder(handle_unknown='ignore'))\n  ]\n)\n\npreprocessor = ColumnTransformer(\n  transformers=[\n    ('num', numerical_transformer, numerical_features),\n    ('cat', categorical_transformer, categorical_features)\n  ]\n)\n\nclassifier = LGBMClassifier(n_estimators=300)\n\nmodel = Pipeline(\n  steps=[\n    ('preprocessor', preprocessor),\n    ('classifier', classifier)\n  ]\n)\nmodel.fit(X_train, y_train)\n\n# create an explainer for the model\nexplainer = dx.Explainer(model, data=X_test, y=y_test, label='lightgbm')\n\n# pack the explainer into a pickle file\nexplainer.dump(open('explainer_lightgbm.pickle', 'wb')) \n```\n\nThen, use `modelStudio` in R:\n\n```r\n# load the explainer from the pickle file\nlibrary(reticulate)\nexplainer \u003c- py_load_object(\"explainer_lightgbm.pickle\", pickle = \"pickle\")\n\n# make a studio for the model\nlibrary(modelStudio)\nmodelStudio(explainer)\n```\n\n\u003c/details\u003e\n\n-------------------------------\n\n## Save \u0026 share\n\nSave `modelStudio` as a HTML file using buttons on the top of the RStudio Viewer\nor with [`r2d3::save_d3_html()`](https://rstudio.github.io/r2d3/articles/publishing.html#save-as-html).\n\n\u003cp align = \"center\", style=\"text-align: center;\"\u003e\n  \u003cimg src=\"man/figures/controls.png\"\u003e\n\u003c/p\u003e\n\n## Citations\n\nIf you use `modelStudio`, please cite our [JOSS article](https://joss.theoj.org/papers/10.21105/joss.01798):\n\n```\n@article{baniecki2019modelstudio,\n  title   = {{modelStudio: Interactive Studio with Explanations for ML Predictive Models}},\n  author  = {Hubert Baniecki and Przemyslaw Biecek},\n  journal = {Journal of Open Source Software},\n  year    = {2019},\n  volume  = {4},\n  number  = {43},\n  pages   = {1798},\n  url     = {https://doi.org/10.21105/joss.01798}\n}\n```\n\nFor a description and evaluation of the Interactive EMA process, refer to our [DAMI article](https://doi.org/10.1007/s10618-023-00924-w):\n\n```\n@article{baniecki2023grammar,\n  title   = {The grammar of interactive explanatory model analysis},\n  author  = {Hubert Baniecki and Dariusz Parzych and Przemyslaw Biecek},\n  journal = {Data Mining and Knowledge Discovery},\n  year    = {2023},\n  pages   = {1--37},\n  url     = {https://doi.org/10.1007/s10618-023-00924-w}\n}\n```\n\n## More resources\n  \n  - Introduction to the plots: [Explanatory Model Analysis: Explore, Explain, and Examine Predictive Models](https://ema.drwhy.ai)\n\n  - Vignettes: [perks and features](https://modelstudio.drwhy.ai/articles/ms-perks-features.html), [R \u0026 Python examples](https://modelstudio.drwhy.ai/articles/ms-r-python-examples.html), [modelStudio in R Markdown HTML](https://modelstudio.drwhy.ai/articles/ms-rmarkdown.html)   \n  \n  - Changelog: [NEWS](https://modelstudio.drwhy.ai/news/index.html)\n    \n  - Conference poster: [ML in PL 2019](https://github.com/ModelOriented/modelStudio/blob/master/misc/MLinPL2019_modelStudio_poster.pdf)\n  \n  \n## Acknowledgments\n\nWork on this package was financially supported by the National Science Centre (Poland) grant `2016/21/B/ST6/02176` and National Centre for Research and Development grant `POIR.01.01.01-00-0328/17`.\n","funding_links":["https://github.com/sponsors/hbaniecki"],"categories":["Python Libraries(sort in alphabeta order)","R","Technical Resources"],"sub_categories":["Evaluation methods","Open Source/Access Responsible AI Software Packages"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FModelOriented%2FmodelStudio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FModelOriented%2FmodelStudio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FModelOriented%2FmodelStudio/lists"}