{"id":19311182,"url":"https://github.com/forestry-labs/distillml","last_synced_at":"2025-04-22T14:30:33.485Z","repository":{"id":38785068,"uuid":"419453556","full_name":"forestry-labs/distillML","owner":"forestry-labs","description":"An R package providing functions for interpreting and distilling machine learning models","archived":false,"fork":false,"pushed_at":"2023-04-19T21:51:31.000Z","size":10233,"stargazers_count":7,"open_issues_count":4,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-02T00:35:33.731Z","etag":null,"topics":["bart","distillation-model","explainable-machine-learning","explainable-ml","interpretability","interpretable-machine-learning","machine-learning","model","random-forest","xgboost"],"latest_commit_sha":null,"homepage":"https://forestry-labs.github.io/distillML","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/forestry-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-20T18:53:12.000Z","updated_at":"2024-11-24T07:13:44.000Z","dependencies_parsed_at":"2024-11-10T00:28:11.130Z","dependency_job_id":"9b39ff7c-31a5-4a3a-aeb9-3469ee123e97","html_url":"https://github.com/forestry-labs/distillML","commit_stats":{"total_commits":180,"total_committers":3,"mean_commits":60.0,"dds":0.4444444444444444,"last_synced_commit":"695e061a5bdf7d61bef0a2d372399b4db533c089"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forestry-labs%2FdistillML","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forestry-labs%2FdistillML/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forestry-labs%2FdistillML/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forestry-labs%2FdistillML/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/forestry-labs","download_url":"https://codeload.github.com/forestry-labs/distillML/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250258744,"owners_count":21400961,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bart","distillation-model","explainable-machine-learning","explainable-ml","interpretability","interpretable-machine-learning","machine-learning","model","random-forest","xgboost"],"created_at":"2024-11-10T00:28:01.930Z","updated_at":"2025-04-22T14:30:28.474Z","avatar_url":"https://github.com/forestry-labs.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![R-CMD-check](https://github.com/forestry-labs/distillML/actions/workflows/check-noncontainerized.yaml/badge.svg)](https://github.com/forestry-labs/distillML/actions/workflows/check-noncontainerized.yaml)\n\n# distillML: Interpretable Machine Learning Methods and Surrogate Model Methods\n\n`distillML` provides several methods for model distillation and\ninterpretability for general black box machine learning models. This\npackage provides implementations of the partial dependence plot (PDP),\nindividual conditional expectation (ICE), and accumulated local effect\n(ALE) methods, which are model-agnostic interpretability methods (work\nwith any supervised machine learning model). This package also provides\na novel method for building a surrogate model that approximates the\nbehavior of its initial algorithm.\n\nBelow, we provide a simple example that outlines how to use this\npackage. For further details on surrogate distillation, advanced\ninterpretability features, or local surrogate methods, see the articles\nprovided on this page.\n\nFor documentation, see:\n\u003chttps://forestry-labs.github.io/distillML/reference/index.html\u003e\n\n## A Simple Example: Predicting Carapace Width of Leptograpsus Crabs\n\nThroughout this section, we provide a tutorial on using the package with\na random forest predictor for the Carapace Width of Leptograpsus Crabs.\nWe demonstrate how to plot PDP, ICE, and ALE curves for machine learning\ninterpretability, and show how to build the surrogate model that\napproximates the behavior of the initial random forest predictor.\n\n### General Prediction Wrapper\n\nFirst we load in the crabs data set. This contains physical measurements\nof several species of crabs collected at Fremantle, West Australia.\n\n``` r\nlibrary(MASS)\nlibrary(distillML)\nlibrary(Rforestry)\nlibrary(ggplot2)\n\nset.seed(491)\n\ndata \u003c- MASS::crabs\nlevels(data$sex) \u003c- list(Male = \"M\", Female = \"F\")\nlevels(data$sp) \u003c- list(Orange = \"O\", Blue = \"B\")\ncolnames(data) \u003c- c(\"Species\",\"Sex\",\"Index\",\"Frontal Lobe\",\n                    \"Rear Width\", \"Carapace Length\",\"Carapace Width\",\n                    \"Body Depth\")\n```\n\nWe can train a random forest to estimate the Carapace Width of the crabs\nbased on the other features. In order to use the interpretability\nfeatures, we must create a `Predictor` class for the estimator we want\nto interpret. This class standardizes the predictions, tracks the\noutcome feature, and stores the training data.\n\n``` r\n# Get training data set\nset.seed(491)\ntest_ind \u003c- sample(1:nrow(data), nrow(data)%/%5)\ntrain_reg \u003c- data[-test_ind,]\ntest_reg \u003c- data[test_ind,]\n\n# Train a random forest on the data set\nforest \u003c- forestry(x=train_reg[,-which(names(train_reg)==\"Carapace Width\")],\n                   y=train_reg[,which(names(train_reg)==\"Carapace Width\")])\n\n# Create a predictor wrapper for the forest\n# this allows us to use a standard wrapper for querying any \n# trained estimator\nforest_predictor \u003c- Predictor$new(model = forest, \n                                  data=train_reg, \n                                  y=\"Carapace Width\",\n                                  task = \"regression\")\n```\n\n### Interpretability Wrapper\n\nOnce we have initialized a `Predictor` object for the forest, we can\npass this to the `Interpreter` class. By default, the `Interpreter`\nclass subsamples the training data to be at most 1000 samples in order\nto speed up computation for interpretabilitiy methods. This class\nprovides the names and classes of the features, the indicies of the\nsampled data points, lists of univariate and bivariate PDP functions,\nand stores additional information for plot settings.\n\n``` r\nforest_interpret \u003c- Interpreter$new(predictor = forest_predictor)\n\nprint(forest_interpret)\n```\n\n    ## \u003cInterpreter\u003e\n    ##   Public:\n    ##     ale.grid: list\n    ##     center.at: list\n    ##     clone: function (deep = FALSE) \n    ##     data.points: 17 59 105 8 18 51 157 37 102 44 119 131 107 75 7 148 60  ...\n    ##     feat.class: factor factor integer numeric numeric numeric numeric\n    ##     features: Species Sex Index Frontal Lobe Rear Width Carapace Lengt ...\n    ##     features.2d: data.frame\n    ##     grid.points: list\n    ##     grid.size: 50\n    ##     initialize: function (predictor = NULL, samples = 1000, data.points = NULL, \n    ##     pdp.1d: list\n    ##     pdp.2d: list\n    ##     predictor: Predictor, R6\n    ##     saved: list\n\nThe PDP functions are stored in two lists, one for univariate PDP\nfunctions and one for bivariate PDP functions.For any feature, we can\nretrieve the pdp function by selecting the entry in the list with that\nfeature name. We can directly use these PDP functions by specifying\nvalues for a specific feature. The functions then return the PDP curve’s\nvalues. For univariate functions, we specify values through a vector of\nvalues. For bivariate functions, we input a dataframe or matrix with two\ncolumns, with each row providing a pair of values and each column\nrepresenting a specific feature.\n\n``` r\n# univariate PDP \none_feat \u003c- train_reg$`Frontal Lobe`[1:10]\npreds_pdp \u003c- forest_interpret$pdp.1d$`Frontal Lobe`(one_feat)\nprint(preds_pdp)\n```\n\n    ##  [1] 34.30249 34.41734 34.44234 34.44921 34.67518 35.00046 35.02284 35.11743\n    ##  [9] 35.47968 35.47968\n\n``` r\n# bivariate PDP\ntwo_feat \u003c- cbind(train_reg$`Frontal Lobe`[1:10], \n                  train_reg$`Rear Width`[1:10])\npreds_pdp_2d \u003c- forest_interpret$pdp.2d$`Frontal Lobe`$`Rear Width`(two_feat)\nprint(preds_pdp_2d)\n```\n\n    ##  [1] 31.86542 32.27242 32.33959 32.35324 33.07650 33.29917 33.40389 34.38326\n    ##  [9] 34.61258 34.78585\n\n### Basic Plotting\n\nFor univariate and bivariate interpretability methods, we can use the\n`plot` method for the Interpreter class. For univariate summaries of the\nmodel’s behavior, we have three main options: PDP, ICE, and ALE curves.\nFor all univariate plots for a feature, `distillML` includes a histogram of \nthe marginal distribution of that feature to show the support.\nTo plot a specific curve for a given set of feature, we simply specify\nthe `method` parameter in `plot` function, as shown below:\n\n``` r\n# plotting PDP functions\nplot(forest_interpret,\n     method = \"pdp\",\n     features = c(\"Frontal Lobe\", \"Rear Width\"))\n```\n\n![](man/figures/pdp.png)\u003c!-- --\u003e\n\n``` r\nplot(forest_interpret,\n     method = \"ice\",\n     features = c(\"Frontal Lobe\", \"Rear Width\"))\n```\n\n![](man/figures/ice.png)\u003c!-- --\u003e\n\n``` r\n## default option (does this without specifying method)\nplot(forest_interpret,\n     method = \"pdp+ice\",\n     features = c(\"Frontal Lobe\", \"Rear Width\"))\n```\n\n![](man/figures/pdp_ice.png)\u003c!-- --\u003e\n\n``` r\nplot(forest_interpret,\n     method = \"ale\",\n     features = c(\"Frontal Lobe\", \"Rear Width\"))\n```\n\n![](man/figures/ale.png)\u003c!-- --\u003e\n\nFor bivariate summary plots, the package provides two distinct methods.\nGiven a continuous and categorical feature, the `plot` function provides\nconditional PDP curves, which separates the mean values based on the\ncategorical feature value. For two continuous features, the `plot`\nfunction provides a PDP heatmap. To input the pairs of features to plot, we\nspecify this in the form of a two-column dataframe of feature names,\nwhere each row represents a single pair.\n\n``` r\nplot(forest_interpret,\n     features.2d = data.frame(feat.1 = c(\"Frontal Lobe\", \"Frontal Lobe\"),\n                              feat.2 = c(\"Sex\", \"Rear Width\")))\n```\n\n    ## $`Frontal Lobe.Sex`\n\n![](man/figures/unnamed-chunk-6-1.png)\u003c!-- --\u003e\n\n    ## \n    ## $`Frontal Lobe.Rear Width`\n\n![](man/figures/unnamed-chunk-6-2.png)\u003c!-- --\u003e\n\nFor more advanced plotting features, such as clustering ICE curves or\nspecifying the number of points plotted, please refer to the article\n“Advanced Plotting Features”.\n\n### Local Surrogates\n\nEven with a heatmap or conditional plots, two dimensional summaries may\nbe difficult to interpret. The function `localSurrogate` provides a\nlocal summary of how changes in a pair of features affect the\npredictions of the model by providing a simple decision tree summary. \nIn the plots below, the left tree represents the \"Frontal Lobe\" and \"Sex\" pair, \nwhile the right tree represents the \"Frontal Lobe\" and \"Rear Width\" pair.\n\n``` r\nlocal.surr \u003c- localSurrogate(forest_interpret,\n                             features.2d = data.frame(feat.1 = c(\"Frontal Lobe\", \n                                                                 \"Frontal Lobe\"),\n                                                      feat.2 = c(\"Sex\", \n                                                                 \"Rear Width\")))\nplot(local.surr$models$`Frontal Lobe.Sex`)\nplot(local.surr$models$`Frontal Lobe.Rear Width`)\n```\n\n![](man/figures/local_surr.png)\u003c!-- --\u003e\n\n\nFor additional details on the `localSurrogate` method, such as\nspecifying the depth or number of trees in the weak learner, please\nrefer to the article “Local Surrogate”.\n\n### Distillation: Creating the Default Surrogate Model\n\nIn this package, we also provide an implementation of a new algorithm,\nwhich creates a linear recombination of the univariate PDP curves to\ngenerate a surrogate model. To do this, we use the `distill` method on\nan interpeter object, which returns a surrogate model. With this\nsurrogate model, we can make predictions, and compare the original\npredictions of the random forest and those of the surrogate model below.\n\n``` r\nforest_surrogate \u003c- distill(forest_interpret)\n\npredictions_forest \u003c- predict(forest,\n                              test_reg[,-which(names(test_reg) == \"Carapace Width\")])\n\n# surrogate predictions are returned as a one-column dataframe\npredictions_surrogate \u003c- predict(forest_surrogate,\n                                 test_reg[,-which(names(test_reg) == \"Carapace Width\")])\n\nplot.comparison \u003c- data.frame(original = predictions_forest,\n                              surrogate = predictions_surrogate[,1])\nggplot(data = plot.comparison, aes(x = original, y = surrogate)) + \n  geom_point() + geom_abline(col = \"red\")\n```\n\n![](man/figures/unnamed-chunk-8-1.png)\u003c!-- --\u003e\n\nFor additional details on creating the distilled surrogate models,\nplease refer to the article “Distillation Methods”.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fforestry-labs%2Fdistillml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fforestry-labs%2Fdistillml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fforestry-labs%2Fdistillml/lists"}