{"id":51588076,"url":"https://github.com/krisrs1128/interpretability_review","last_synced_at":"2026-07-11T12:31:32.363Z","repository":{"id":221773393,"uuid":"749579900","full_name":"krisrs1128/interpretability_review","owner":"krisrs1128","description":"Code accompanying a review article on interpretability and XAI. Includes examples for both simple (sparse regression) and sophisticated (concept bottlenecks) approaches, using notebooks that can be run in a few minutes.","archived":false,"fork":false,"pushed_at":"2024-08-17T22:39:37.000Z","size":31429,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-08-18T21:32:15.955Z","etag":null,"topics":["concepts","data-science","interpretability","review","xai"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/krisrs1128.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-29T00:44:24.000Z","updated_at":"2024-08-17T22:39:40.000Z","dependencies_parsed_at":"2024-02-10T01:23:19.761Z","dependency_job_id":"1c5c87af-b484-4e4c-830f-a874e544bf0b","html_url":"https://github.com/krisrs1128/interpretability_review","commit_stats":{"total_commits":65,"total_committers":1,"mean_commits":65.0,"dds":0.0,"last_synced_commit":"6eb0352383c9978fa55983c4379b68202946d9a8"},"previous_names":["krisrs1128/interpretability_review"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/krisrs1128/interpretability_review","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisrs1128%2Finterpretability_review","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisrs1128%2Finterpretability_review/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisrs1128%2Finterpretability_review/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisrs1128%2Finterpretability_review/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/krisrs1128","download_url":"https://codeload.github.com/krisrs1128/interpretability_review/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisrs1128%2Finterpretability_review/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35362871,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-11T02:00:05.354Z","response_time":104,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concepts","data-science","interpretability","review","xai"],"created_at":"2026-07-11T12:31:31.913Z","updated_at":"2026-07-11T12:31:32.358Z","avatar_url":"https://github.com/krisrs1128.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n### Interpretability Review\n\nThis repository contains code accompanying [Data Science Principles for Interpretable and Explainable AI](https://doi.org/10.6339/24-JDS1150). This review article gives an overview of\ninterpretability research through a statistical lens.\n\n![Summary of XAI Techniques](data/xai-summary.png)\n\nTo reproduce the figures in the main text,\nyou can run these notebooks. None should take longer then 5 minutes to complete.\n\n* `interpretable.Rmd`: Code for the sparse logistic regression and decision tree models on both the raw and featurized versions of microbiome trajectory data. This creates Figure 1 in the text.\n* `transformer.ipynb`: Run a transformer on the trajectory data. This trains the model that is analyzed using global embeddings and integrated gradients in the next steps.\n* `concept_bottlneck.ipynb`: Run a concept bottleneck model on trajectory data.\n* `embeddings.ipynb`: Extract and save the embeddings associated with the trained transformer model.\n* `embeddings.Rmd`: Visualize the embeddings saved by `embeddings.ipynb`. This creates Figure 3. \n* `integrated_gradients.ipynb`: Save the integrated gradients for a subset of samples.\n* `integrated_gradients.Rmd`: Visualize the integrated gradient estimates saved by `integrated_gradients.ipynb`. This creates Figure 4.\n\n### Data and Environment Setup\n\nThe data used in the case study are generated in the notebook\n`generate/concept.Rmd`. They are also saved in the the [`data`\nfolder](https://github.com/krisrs1128/interpretability_review/tree/main/data) of\nthis repository, in case you want to run the modeling and interpretation code\ndirectly.\n\nIf you are running this code on your own laptop, you will need to setup your\nenvironment with the following packages:\n\n* R: `LaplacesDemon`, `RcppEigen`, `broom`, `ggdendro`, `ggrepel`, `glmnetUtils`, `glue`, `gsignal`, `patchwork`, `scico`, `sparsepca`, `tictoc`, `tidymodels`, `tidyverse`\n* python: `captum` `lightning`, `numpy`, `pandas`, `tensorboard`, `torch`, `transformer`, \n\n`LaplacesDemon` and `gsignal` are only needed to regenerate the data, and you\ncan ignore them if you want to use the files that have already been saved here.\nMany of these packages (e.g., `ggrepel`, `ggdendro`, `patchwork`, `scico`) are\nonly used to refine the visualizations, and you can safely omit them if you are\nhappy with ggplot2 defaults.\n\nIn R, you can install these packages from CRAN:\n```\npkgs \u003c- c(\"LaplacesDemon\", \"RcppEigen\", \"broom\", \"ggdendro\", \"ggrepel\", \"glmnetUtils\", \"glue\", \"gsignal\", \"patchwork\", \"scico\", \"sparsepca\", \"tictoc\", \"tidymodels\", \"tidyverse\")\ninstall.packages(pkgs)\n```\n\nFor python, you can create a conda environment with these packages:\n\n```\nconda create -n interpretability python=3.12\nconda activate interpretability\n\nconda install -y conda-forge::lightning\nconda install -y conda-forge::pandas\nconda install -y conda-forge::tensorboard\nconda install -y conda-forge::transformers\nconda install -y pytorch::captum\n```\n\nAll python notebooks are assumed to be run from folder they are saved in, while\nthe R notebooks are assumed to be run with the repository root\n(`interpretability_review`) as the working directory.\n\n### Contact\n\nIf you have any questions, don't hesitate to create an issue or reach out to\n[ksankaran@wisc.edu](mailto:ksankaran@wisc.edu)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrisrs1128%2Finterpretability_review","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkrisrs1128%2Finterpretability_review","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrisrs1128%2Finterpretability_review/lists"}