{"id":13737213,"url":"https://github.com/csinva/hierarchical-dnn-interpretations","last_synced_at":"2025-06-13T02:41:59.707Z","repository":{"id":57407905,"uuid":"133954255","full_name":"csinva/hierarchical-dnn-interpretations","owner":"csinva","description":"Using / reproducing ACD from the paper \"Hierarchical interpretations for neural network predictions\" 🧠 (ICLR 2019)","archived":false,"fork":false,"pushed_at":"2021-08-25T12:27:37.000Z","size":51020,"stargazers_count":128,"open_issues_count":2,"forks_count":23,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-05-13T09:11:33.319Z","etag":null,"topics":["acd","ai","artificial-intelligence","convolutional-neural-networks","data-science","deep-learning","deep-neural-networks","explainability","explainable-ai","feature-importance","iclr","interpretability","interpretation","jupyter-notebook","machine-learning","ml","neural-network","python","pytorch","statistics"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1806.05337","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csinva.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"citation.bib","codeowners":null,"security":null,"support":null}},"created_at":"2018-05-18T12:54:43.000Z","updated_at":"2024-12-25T09:35:49.000Z","dependencies_parsed_at":"2022-09-13T04:50:23.666Z","dependency_job_id":null,"html_url":"https://github.com/csinva/hierarchical-dnn-interpretations","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/csinva/hierarchical-dnn-interpretations","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fhierarchical-dnn-interpretations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fhierarchical-dnn-interpretations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fhierarchical-dnn-interpretations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fhierarchical-dnn-interpretations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csinva","download_url":"https://codeload.github.com/csinva/hierarchical-dnn-interpretations/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Fhierarchical-dnn-interpretations/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259568153,"owners_count":22877858,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["acd","ai","artificial-intelligence","convolutional-neural-networks","data-science","deep-learning","deep-neural-networks","explainability","explainable-ai","feature-importance","iclr","interpretability","interpretation","jupyter-notebook","machine-learning","ml","neural-network","python","pytorch","statistics"],"created_at":"2024-08-03T03:01:37.639Z","updated_at":"2025-06-13T02:41:59.651Z","avatar_url":"https://github.com/csinva.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e Hierarchical neural-net interpretations (ACD) 🧠\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e Produces hierarchical interpretations for a single prediction made by a pytorch neural network. Official code for \u003ci\u003eHierarchical interpretations for neural network predictions\u003c/i\u003e (ICLR 2019 \u003ca href=\"https://openreview.net/pdf?id=SkEqro0ctQ\"\u003epdf\u003c/a\u003e). \u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/license-mit-blue.svg\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/python-3.6--3.8-blue\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/pytorch-1.0%2B-blue\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/checks-status/csinva/hierarchical-dnn-interpretations/master\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/v/acd?color=orange\"\u003e\n  \u003cimg src=\"https://static.pepy.tech/personalized-badge/acd?period=total\u0026units=none\u0026left_color=gray\u0026right_color=orange\u0026left_text=downloads\"\u003e\n\u003c/p\u003e  \n\u003cp align=\"center\"\u003e\n\t\u003ca href=\"https://csinva.io/hierarchical-dnn-interpretations/\"\u003eDocumentation\u003c/a\u003e •\n  \u003ca href=\"https://github.com/csinva/hierarchical-dnn-interpretations/tree/master/reproduce_figs\"\u003eDemo notebooks\u003c/a\u003e\n\u003c/p\u003e  \n\u003cp align=\"center\"\u003e\n\t\u003ci\u003eNote: this repo is actively maintained. For any questions please file an issue.\u003c/i\u003e\n\u003c/p\u003e\n\n\n![](https://csinva.io/hierarchical-dnn-interpretations/intro.svg?sanitize=True)\n\n\n\n# examples/documentation\n\n- **installation**: `pip install acd` (or clone and run `python setup.py install`)\n- **examples**: the [reproduce_figs](https://github.com/csinva/hierarchical-dnn-interpretations/tree/master/reproduce_figs) folder has notebooks with many demos\n- **src**: the [acd](acd) folder contains the source for the method implementation\n- allows for different types of interpretations by changing hyperparameters (explained in examples)\n- all required data/models/code for reproducing are included in the [dsets](dsets) folder\n\n| Inspecting NLP sentiment models    | Detecting adversarial examples      | Analyzing imagenet models           |\n| ---------------------------------- | ----------------------------------- | ----------------------------------- |\n| ![](reproduce_figs/figs/fig_2.png) | ![](reproduce_figs/figs/fig_s3.png) | ![](reproduce_figs/figs/fig_s2.png) |\n\n\n# notes on using ACD on your own data\n- the current CD implementation often works out-of-the box, especially for networks built on common layers, such as alexnet/vgg/resnet. However, if you have custom layers or layers not accessible in `net.modules()`, you may need to write a custom function to iterate through some layers of your network (for examples see `cd.py`). \n- to use baselines such build-up and occlusion, replace the pred_ims function by a function, which gets predictions from your model given a batch of examples.\n\n\n# related work\n\n- CDEP (ICML 2020 [pdf](https://arxiv.org/abs/1909.13584), [github](https://github.com/laura-rieger/deep-explanation-penalization)) - penalizes CD / ACD scores during training to make models generalize better\n- TRIM (ICLR 2020 workshop [pdf](https://arxiv.org/abs/2003.01926), [github](https://github.com/csinva/transformation-importance)) - using simple reparameterizations, allows for calculating disentangled importances to transformations of the input (e.g. assigning importances to different frequencies)\n- PDR framework (PNAS 2019 [pdf](https://arxiv.org/abs/1901.04592)) - an overarching framewwork for guiding and framing interpretable machine learning\n- DAC (arXiv 2019 [pdf](https://arxiv.org/abs/1905.07631), [github](https://github.com/csinva/disentangled-attribution-curves)) - finds disentangled interpretations for random forests\n- Baseline interpretability methods - the file `scores/score_funcs.py` also contains simple pytorch implementations of [integrated gradients](https://arxiv.org/abs/1703.01365) and the simple interpration technique `gradient * input`\n\n# reference\n\n- feel free to use/share this code openly\n- if you find this code useful for your research, please cite the following:\n\n ```r\n@inproceedings{\n    singh2019hierarchical,\n    title={Hierarchical interpretations for neural network predictions},\n    author={Chandan Singh and W. James Murdoch and Bin Yu},\n    booktitle={International Conference on Learning Representations},\n    year={2019},\n    url={https://openreview.net/forum?id=SkEqro0ctQ},\n}\n ```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsinva%2Fhierarchical-dnn-interpretations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsinva%2Fhierarchical-dnn-interpretations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsinva%2Fhierarchical-dnn-interpretations/lists"}