{"id":32206153,"url":"https://github.com/bdwilliamson/vimp","last_synced_at":"2026-02-20T16:02:04.640Z","repository":{"id":49808873,"uuid":"60721402","full_name":"bdwilliamson/vimp","owner":"bdwilliamson","description":"Perform inference on algorithm-agnostic variable importance","archived":false,"fork":false,"pushed_at":"2025-09-25T18:54:00.000Z","size":16126,"stargazers_count":24,"open_issues_count":0,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-12-08T23:20:44.836Z","etag":null,"topics":["machine-learning","nonparametric-statistics","statistical-inference","variable-importance"],"latest_commit_sha":null,"homepage":"http://bdwilliamson.github.io/vimp/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bdwilliamson.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-06-08T18:38:49.000Z","updated_at":"2025-09-25T18:47:36.000Z","dependencies_parsed_at":"2025-07-23T18:07:53.004Z","dependency_job_id":"251b4265-f92f-4133-9cf0-ebbc401ccb8b","html_url":"https://github.com/bdwilliamson/vimp","commit_stats":{"total_commits":997,"total_committers":5,"mean_commits":199.4,"dds":"0.14142427281845538","last_synced_commit":"141fd0315282c63fbbd7ebf1356cda7160dc8ea8"},"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"purl":"pkg:github/bdwilliamson/vimp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdwilliamson%2Fvimp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdwilliamson%2Fvimp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdwilliamson%2Fvimp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdwilliamson%2Fvimp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bdwilliamson","download_url":"https://codeload.github.com/bdwilliamson/vimp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bdwilliamson%2Fvimp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29656589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-20T09:27:29.698Z","status":"ssl_error","status_checked_at":"2026-02-20T09:26:12.373Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","nonparametric-statistics","statistical-inference","variable-importance"],"created_at":"2025-10-22T05:05:08.708Z","updated_at":"2026-02-20T16:02:04.628Z","avatar_url":"https://github.com/bdwilliamson.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"# R/`vimp`: inference on algorithm-agnostic variable importance \u003cimg src=\"man/figures/logo.png\" align=\"right\" width=\"120px\"/\u003e\n\n \u003c!-- badges: start --\u003e\n[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/vimp)](https://cran.r-project.org/package=vimp)\n[![R-CMD-check](https://github.com/bdwilliamson/vimp/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/bdwilliamson/vimp/actions/workflows/R-CMD-check.yaml)\n[![Coverage status](https://codecov.io/gh/bdwilliamson/vimp/branch/master/graph/badge.svg)](https://codecov.io/github/bdwilliamson/vimp?branch=master)\n[![CRAN downloads](https://cranlogs.r-pkg.org/badges/vimp)](https://CRAN.R-project.org/package=vimp)\n[![Project Status: Active – The project has reached a stable, usable state and is being actively developed.](http://www.repostatus.org/badges/latest/active.svg)](http://www.repostatus.org/#active)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\u003c!-- badges: end --\u003e\n\n**Software authors:** [Brian Williamson](https://bdwilliamson.github.io/), [Jean Feng](https://jeanfeng.com/), and [Charlie Wolock](https://cwolock.github.io/)\n\n**Methodology authors:** [Brian Williamson](https://bdwilliamson.github.io/), [Peter Gilbert](https://www.fredhutch.org/en/faculty-lab-directory/gilbert-peter.html), [Noah Simon](http://faculty.washington.edu/nrsimon/), [Marco Carone](http://faculty.washington.edu/mcarone/about.html), [Jean Feng](http://www.jeanfeng.com/)\n\n**Python package:** https://github.com/bdwilliamson/vimpy\n\n## Introduction\n\nIn predictive modeling applications, it is often of interest to determine the relative contribution of subsets of features in explaining an outcome; this is often called variable importance. It is useful to consider variable importance as a function of the unknown, underlying data-generating mechanism rather than the specific predictive algorithm used to fit the data. This package provides functions that, given fitted values from predictive algorithms, compute algorithm-agnostic estimates of population variable importance, along with asymptotically valid confidence intervals for the true importance and hypothesis tests of the null hypothesis of zero importance.\n\nSpecifically, the types of variable importance supported by `vimp` include: difference in population classification accuracy, difference in population area under the receiver operating characteristic curve, difference in population deviance, difference in population R-squared.\n\nMore detail may be found in our papers on [R-squared-based variable importance](https://doi.org/10.1111/biom.13392),  [general variable importance](https://doi.org/10.1080/01621459.2021.2003200), and [general Shapley-based variable importance](http://proceedings.mlr.press/v119/williamson20a.html).\n\nThis method works on low-dimensional and high-dimensional data.\n\n## Issues\n\nIf you encounter any bugs or have any specific feature requests, please [file an issue](https://github.com/bdwilliamson/vimp/issues).\n\n## R installation\n\nYou may install a stable release of `vimp` from [CRAN](https://cran.r-project.org/web/packages/vimp/index.html) via `install.packages(\"vimp\")`. You may also install a stable release of `vimp` from GitHub via [`devtools`](https://www.rstudio.com/products/rpackages/devtools/) by running the following code (replace `v2.1.0` with the tag for the specific release you wish to install):\n\n```r\n## install.packages(\"devtools\") # only run this line if necessary\ndevtools::install_github(repo = \"bdwilliamson/vimp@v2.1.0\")\n```\n\nYou may install a development release of `vimp` from GitHub via [`devtools`](https://www.rstudio.com/products/rpackages/devtools/) by running the following code:\n\n```r\n## install.packages(\"devtools\") # only run this line if necessary\ndevtools::install_github(repo = \"bdwilliamson/vimp\")\n```\n\n## Example\n\nThis example shows how to use `vimp` in a simple setting with simulated data, using `SuperLearner` to estimate the conditional mean functions and specifying the importance measure of interest as the R-squared-based measure. For more examples and detailed explanation, please see the [vignette](https://bdwilliamson.github.io/vimp/introduction-to-vimp.html).\n\n```r\n# load required functions and libraries\nlibrary(\"SuperLearner\")\nlibrary(\"vimp\")\nlibrary(\"xgboost\")\nlibrary(\"glmnet\")\n\n# -------------------------------------------------------------\n# problem setup\n# -------------------------------------------------------------\n# set up the data\nn \u003c- 100\np \u003c- 2\ns \u003c- 1 # desire importance for X_1\nx \u003c- as.data.frame(replicate(p, runif(n, -1, 1)))\ny \u003c- (x[,1])^2*(x[,1]+7/5) + (25/9)*(x[,2])^2 + rnorm(n, 0, 1)\n\n# -------------------------------------------------------------\n# get variable importance!\n# -------------------------------------------------------------\n# set up the learner library, consisting of the mean, boosted trees,\n# elastic net, and random forest\nlearner.lib \u003c- c(\"SL.mean\", \"SL.xgboost\", \"SL.glmnet\", \"SL.randomForest\")\n# get the variable importance estimate, SE, and CI\n# I'm using only 2 cross-validation folds to make things run quickly; in practice, you should use more\nset.seed(20231213)\nvimp \u003c- vimp_rsquared(Y = y, X = x, indx = 1, V = 2)\n```\n\n## Citation\n\nAfter using the `vimp` package, please cite the following (for R-squared-based variable importance):\n\n```\n    @article{williamson2020,\n      author={Williamson, BD and Gilbert, PB and Carone, M and Simon, R},\n      title={Nonparametric variable importance assessment using machine learning techniques},\n      journal={Biometrics},\n      year={2020},\n      doi={10.1111/biom.13392}\n    }\n```\n\nor the following (for general variable importance parameters):\n\n```\n  @article{williamson2021,\n    author={Williamson, BD and Gilbert, PB and Simon, NR and Carone, M},\n    title={A general framework for inference on algorithm-agnostic variable importance},\n    journal={Journal of the American Statistical Association},\n    year={2021},\n    doi={10.1080/01621459.2021.2003200}\n  }\n```\n\nor the following (for Shapley-based variable importance):\n\n```\n  @inproceedings{williamson2020,\n    title={Efficient nonparametric statistical inference on population feature importance using {S}hapley values},\n    author={Williamson, BD and Feng, J},\n    year={2020},\n    booktitle={Proceedings of the 37th International Conference on Machine Learning},\n    volume={119},\n    pages={10282--10291},\n    series = {Proceedings of Machine Learning Research},\n    URL = {http://proceedings.mlr.press/v119/williamson20a.html}\n}\n```\n\n## License\n\nThe contents of this repository are distributed under the MIT license. See below for details:\n\n```\nMIT License\n\nCopyright (c) [2018-present] [Brian D. Williamson]\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n```\n\n## Logo\n\nThe logo was created using [hexSticker](https://github.com/GuangchuangYu/hexSticker) and [lisa](https://github.com/tyluRp/lisa). Many thanks to the maintainers of these packages and the [Color Lisa](http://colorlisa.com/) team.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbdwilliamson%2Fvimp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbdwilliamson%2Fvimp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbdwilliamson%2Fvimp/lists"}