{"id":13711286,"url":"https://github.com/stefanoteso/awesome-explanatory-supervision","last_synced_at":"2025-05-06T20:32:37.891Z","repository":{"id":40996071,"uuid":"334714853","full_name":"stefanoteso/awesome-explanatory-supervision","owner":"stefanoteso","description":"List of relevant resources for machine learning from explanatory supervision","archived":false,"fork":false,"pushed_at":"2024-07-09T09:05:36.000Z","size":195,"stargazers_count":152,"open_issues_count":0,"forks_count":16,"subscribers_count":10,"default_branch":"main","last_synced_at":"2024-11-08T10:02:01.361Z","etag":null,"topics":["active-learning","awesome-list","explainable-ai","explanatory-supervision","interactive-learning","machine-learning","machine-teaching"],"latest_commit_sha":null,"homepage":"","language":"TeX","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stefanoteso.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-31T17:30:29.000Z","updated_at":"2024-09-10T06:04:57.000Z","dependencies_parsed_at":"2024-05-23T05:06:26.664Z","dependency_job_id":"2cddd22f-f840-4789-9865-527b6d3c8d5e","html_url":"https://github.com/stefanoteso/awesome-explanatory-supervision","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanoteso%2Fawesome-explanatory-supervision","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanoteso%2Fawesome-explanatory-supervision/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanoteso%2Fawesome-explanatory-supervision/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanoteso%2Fawesome-explanatory-supervision/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stefanoteso","download_url":"https://codeload.github.com/stefanoteso/awesome-explanatory-supervision/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224528381,"owners_count":17326353,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["active-learning","awesome-list","explainable-ai","explanatory-supervision","interactive-learning","machine-learning","machine-teaching"],"created_at":"2024-08-02T23:01:06.578Z","updated_at":"2025-05-06T20:32:37.539Z","avatar_url":"https://github.com/stefanoteso.png","language":"TeX","funding_links":[],"categories":["ML Sub-fields"],"sub_categories":["Explainability, Interpretability and Fairness"],"readme":"# Awesome Explanatory Supervision [![Awesome](figures/awesome.svg)](https://github.com/stefanoteso/awesome-explanatory-supervision)\n\nOverview of literature on learning from supervision on the model's *explanations*.  A `.bib` file of the papers below can be downloaded [here](https://raw.githubusercontent.com/stefanoteso/awesome-explanatory-supervision/main/explanatory-supervision.bib).\n\n\n**Warning**: permanent WIP.\n\n\nDid we miss a relevant paper?  Please submit a new entry in the following format:\n\n```markdown\n- **An Artificially-intelligent Means to Escape Discreetly from the Departmental Holiday Party; guide for the socially awkward**\n  Eve Armstrong; arXiv 2020 [paper](https://arxiv.org/abs/2003.14169)\n  `Notes: it is a joke;  a pretty good joke actually.`\n```\n\n\n### Table of Contents\n- [Online Resources](#online-resources)\n- [Passive Learning](#passive-learning)\n- [Interactive Learning](#interactive-learning)\n- [Reinforcement Learning](#reinforcement-learning)\n- [Distillation](#distillation)\n- [Regularization without Supervision](#regularization-without-supervision)\n- [Machine Teaching](#machine-teaching)\n- [Applications](#applications)\n- [Related Works](#related-works)\n- [Resources](#resources)\n\n----\n\n\n### [Online Resources](#online-resources)\n\n- Tutorial on Explanations in Interactive Machine Learning at AAAI-22 [website](https://sites.google.com/view/aaai22-ximl-tutorial/home)\n  `Notes: includes recording`.\n\n----\n\n\n### [Passive Learning](#content)\n\nApproaches that supervise the model's explanations.\n\n- **Rationalizing Neural Predictions**\n  Tao Lei, Regina Barzilay, Tommi Jaakkola; EMNLP 2016 [paper](https://www.aclweb.org/anthology/D16-1011.pdf) [code](github.com/taolei87/rcnn)\n  `Notes: they learn an \"explanation module\" for text classificaiton from explanatory supervision, namely rationales.`\n\n- **Right for the right reasons: training differentiable models by constraining their explanations**\n  Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez; IJCAI 2017 [paper](https://www.ijcai.org/Proceedings/2017/0371.pdf) [code](https://github.com/dtak/rrr)\n\n- **e-SNLI: natural language inference with natural language explanations**\n  Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom; NeurIPS 2018 [paper](https://papers.nips.cc/paper/2018/file/4c7a167bb329bd92580a99ce422d6fa6-Paper.pdf) [code](https://github.com/OanaMariaCamburu/e-SNLI)\n\n- **Tell me where to look: Guided attention inference network**\n  Kunpeng Li, Ziyan Wu, Kuan-Chuan Peng, Jan Ernst, Yun Fu; CVPR 2018 [paper](http://openaccess.thecvf.com/content_cvpr_2018/papers/Li_Tell_Me_Where_CVPR_2018_paper.pdf)\n\n- **e-SNLI: Natural Language Inference with Natural Language Explanations**\n  Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, Phil Blunsom; NeurIPS 2018 [paper](https://proceedings.neurips.cc/paper_files/paper/2018/file/4c7a167bb329bd92580a99ce422d6fa6-Paper.pdf) [code](https://github.com/OanaMariaCamburu/e-SNLI)\n\n- **Learning credible models**\n  Jiaxuan Wang, Jeeheh Oh, Haozhu Wang, and Jenna Wiens; KDD 2018 [paper](https://dl.acm.org/doi/pdf/10.1145/3219819.3220070) [code](https://github.com/nathanwang000/credible_learning)\n\n- **Not Using the Car to See the Sidewalk--Quantifying and Controlling the Effects of Context in Classification and Segmentation**\n  Rakshith Shetty, Bernt Schiele, Mario Fritz; CVPR 2019 [paper](http://openaccess.thecvf.com/content_CVPR_2019/papers/Shetty_Not_Using_the_Car_to_See_the_Sidewalk_--_Quantifying_CVPR_2019_paper.pdf)\n  `Notes: not exactly about explanations, learns from ground-truth object annotations.`\n\n- **Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded**\n  Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, and Devi Parikh; ICCV 2019 [pdf](https://openaccess.thecvf.com/content_ICCV_2019/papers/Selvaraju_Taking_a_HINT_Leveraging_Explanations_to_Make_Vision_and_Language_ICCV_2019_paper.pdf)\n\n- **Learning credible deep neural networks with rationale regularization**\n  Mengnan Du, Ninghao Liu, Fan Yang, Xia Hu; ICDM 2019 [paper](https://arxiv.org/pdf/1908.05601)\n\n- **Deriving Machine Attention from Human Rationales**\n  Yujia Bao, Shiyu Chang, Mo Yu, and Regina Barzilay; ACL 2019 [paper](https://www.aclweb.org/anthology/D18-1216.pdf) [code](https://github.com/YujiaBao/R2A)\n\n- **TED: Teaching AI to explain its decisions**\n  Michael Hind, Dennis Wei, Murray Campbell, Noel Codella, Amit Dhurandhar, Aleksandra Mojsilović, Karthikeyan Ramamurthy, Kush Varshney; AIES 2019 [paper](https://arxiv.org/pdf/1811.04896.pdf)\n\n- **Incorporating Priors with Feature Attribution on Text Classification**\n  Frederick Liu, Besim Avci; ACL 2019 [paper](https://aclanthology.org/P19-1631.pdf)\n\n- **Saliency Learning: Teaching the Model Where to Pay Attention**\n  Reza Ghaeini, Xiaoli Fern, Hamed Shahbazi, Prasad Tadepalli; NAACL 2019 [paper](https://aclanthology.org/N19-1404.pdf)\n\n- **Do Human Rationales Improve Machine Explanations?**\n  Julia Strout, Ye Zhang, Raymond Mooney; ACL Workshop BlackboxNLP 2019 [paper](https://www.aclweb.org/anthology/W19-4807.pdf)\n\n- **CARE: Class attention to regions of lesion for classification on imbalanced data**\n  Jiaxin Zhuang, Jiabin Cai, Ruixuan Wang, Jianguo Zhang, Weishi Zheng; International Conference on Medical Imaging with Deep Learning, 2019. [paper](http://proceedings.mlr.press/v102/zhuang19a/zhuang19a.pdf)\n\n- **GradMask: Reduce Overfitting by Regularizing Saliency**\n  Becks Simpson, Francis Dutil, Yoshua Bengio, Joseph Paul Cohen; International Conference on Medical Imaging with Deep Learning, 2019. [paper](https://openreview.net/pdf?id=Syx2z2aMqE)\n\n- **Learning Global Transparent Models Consistent with Local Contrastive Explanations**\n  Tejaswini Pedapati, Avinash Balakrishnan, Karthikeyan Shanmugam, Amit Dhurandhar; NeurIPS 2020 [paper](https://proceedings.neurips.cc/paper/2020/file/24aef8cb3281a2422a59b51659f1ad2e-Paper.pdf)\n\n- **Model Agnostic Multilevel Explanations**\n  Karthikeyan Natesan Ramamurthy, Bhanukiran Vinzamuri, Yunfeng Zhang, Amit Dhurandhar; NeurIPS 2020 [paper](https://proceedings.neurips.cc/paper/2020/file/426f990b332ef8193a61cc90516c1245-Paper.pdf)\n  `Notes: implicitly learns to generalize across multiple local explanations.`\n\n- **Interpretations are useful: penalizing explanations to align neural networks with prior knowledge**\n  Laura Rieger, Chandan Singh, William Murdoch, Bin Yu; ICML 2020 [paper](http://proceedings.mlr.press/v119/rieger20a/rieger20a.pdf) [code](https://github.com/laura-rieger/deep-explanation-penalization)\n\n- **Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting**\n  Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, William Gan, Joseph Gonzalez, Marcus Rohrbach; ICLR 2020 [paper](https://openreview.net/pdf?id=tHgJoMfy6nI) [code](https://github.com/SaynaEbrahimi/Remembering-for-the-Right-Reasons)\n  `Notes: uses saliency guided replay for continual learning.`\n\n- **Learning to Faithfully Rationalize by Construction**\n  Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, Byron Wallace. ACL 2020 [paper](https://www.aclweb.org/anthology/2020.acl-main.409.pdf) [code](https://github.com/successar/FRESH)\n\n- **Reflective-Net: Learning from Explanations**\n  Johannes Schneider, Michalis Vlachos; arXiv 2020 [paper](https://arxiv.org/pdf/2011.13986.pdf)\n\n- **Learning Interpretable Concept-based Models with Human Feedback**\n  Isaac Lage, Finale Doshi-Velez; arXiv 2020 [paper](https://arxiv.org/pdf/2012.02898)\n  `Notes: incrementally acquires side-information about per-concept feature dependencies; side-information is per-concept, not per-instance.`\n\n- **Improving performance of deep learning models with axiomatic attribution priors and expected gradients**\n  Gabriel Erion, Joseph D. Janizek, Pascal Sturmfels, Scott Lundberg, Su-In Lee; Nature Machine Intelligence 2019 [paper](https://www.nature.com/articles/s42256-021-00343-w) [preprint](https://arxiv.org/pdf/1906.10670) [code](github.com/suinleelab/attributionpriors)\n\n- **GLocalX-From Local to Global Explanations of Black Box AI Models**\n  Mattia Setzu, Riccardo Guidotti, Anna Monreale, Franco Turini, Dino Pedreschi, and Fosca Giannotti; Artificial Intelligence 2021 [page](https://www.sciencedirect.com/science/article/pii/S0004370221000084) [code](https://github.com/msetzu/glocalx)\n  `Notes: converts a set of local explanations to a global explanation / white-box model.`\n\n- **IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammography**\n  Alina Barnett, Fides Schwartz, Chaofan Tao, Chaofan Chen, Yinhao Ren, Joseph Lo, Cynthia Rudin; Nature Machine Intelligence 2021 [paper](https://arxiv.org/pdf/2103.12308) [code](https://github.com/alinajadebarnett/iaiabl)\n\n- **Debiasing Concept-based Explanations with Causal Analysis**\n  Mohammad Taha Bahadori, and David E. Heckerman; ICLR 2021 [paper](https://openreview.net/pdf?id=6puUoArESGp)\n\n- **Teaching with Commentaries**\n  Aniruddh Raghu, Maithra Raghu, Simon Kornblith, David Duvenaud, and Geoffrey Hinton; ICLR 2021 [paper](https://openreview.net/pdf?id=4RbdgBh9gE) [code](github.com/googleinterns/commentaries)\n\n- **Saliency is a possible red herring when diagnosing poor generalization**\n  Joseph Viviano, Becks Simpson, Francis Dutil, Yoshua Bengio, Joseph Paul Cohen; ICLR 2021 [paper](https://openreview.net/pdf?id=c9-WeM-ceB) [code](https://github.com/josephdviviano/saliency-red-herring)\n\n- **Towards Robust Classification Model by Counterfactual and Invariant Data Generation**\n  Chun-Hao Chang, George Alexandru Adam, Anna Goldenberg; CVPR 2021 [paper](https://openaccess.thecvf.com/content/CVPR2021/papers/Chang_Towards_Robust_Classification_Model_by_Counterfactual_and_Invariant_Data_Generation_CVPR_2021_paper.pdf) [code](https://github.com/zzzace2000/robust_cls_model)\n\n- **Global Explanations with Decision Rules: a Co-learning Approach**\n  Géraldin Nanfack, Paul Temple, Benoît Frénay1; UAI 2021 [paper](https://proceedings.mlr.press/v161/nanfack21a/nanfack21a.pdf) [code](https://github.com/gerald4/Co-learning_with_STruGMA)\n\n- **Explain and Predict, and then Predict Again**\n  Zijian Zhang, Koustav Rudra, Avishek Anand; WSDM 2021 [paper](https://arxiv.org/pdf/2101.04109) [code](https://github.com/JoshuaGhost/expred)\n\n- **Explanation-Based Human Debugging of NLP Models: A Survey**\n  Piyawat Lertvittayakumjorn, Francesca Toni; arXiv 2021 [paper](https://arxiv.org/pdf/2104.15135)\n\n- **When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data**\n  Peter Hase, Mohit Bansal; arXiv 2021 [paper](https://arxiv.org/pdf/2102.02201.pdf) [code](https://github.com/peterbhase/ExplanationRoles)\n\n- **Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience**\n  George Chrysostomou, Nikolaos Aletras; arXiv 2021 [paper](https://arxiv.org/pdf/2108.13759) [code](https://github.com/GChrysostomou/saloss)\n\n- **Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updates**\n  Xiaochuang Han, Yulia Tsvetkov; arXiv 2021 [paper](https://arxiv.org/pdf/2110.03212.pdf) [code](https://github.com/xhan77/influence-tuning)\n\n- **Saliency Guided Experience Packing for Replay in Continual Learning**\n  Gobinda Saha, Kaushik Roy; arXiv 2021 [paper](https://arxiv.org/pdf/2109.04954)\n  `Notes: leverages saliency for experience replay in continual learning.`\n\n- **What to Learn, and How: Toward Effective Learning from Rationales**\n  Samuel Carton, Surya Kanoria, Chenhao Tan; arXiv 2021 [paper](https://arxiv.org/pdf/2112.00071)\n\n- **Supervising Model Attention with Human Explanations for Robust Natural Language Inference**\n  Joe Stacey, Yonatan Belinkov, Marek Rei; AAAI 2022 [paper](https://arxiv.org/pdf/2104.08142.pdf) [code](https://github.com/joestacey/NLI_with_a_human_touch)\n\n- **Finding and removing Clever Hans: Using explanation methods to debug and improve deep models**\n  Christopher Anders, Leander Weber, David Neumann, Wojciech Samek, Klaus-Robert Müller, Klaus-Robert, Sebastian Lapuschkin; Information Fusion 2022 [paper](https://www.sciencedirect.com/science/article/pii/S1566253521001573) [code](https://github.com/virelay/virelay) [code](https://github.com/virelay/corelay)\n\n- **Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features**\n  Haohan Wang, Zeyi Huang, Hanlin Zhang, Eric P. Xing; UAI 2022 [paper](https://arxiv.org/pdf/2111.03740) [code](https://github.com/OoDBag/WR)\n\n- **A survey on improving NLP models with human explanations**\n  Mareike Hartmann, Daniel Sonntag; arXiv 2022 [paper](https://arxiv.org/pdf/2204.08892.pdf)\n\n- **VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives**\n  Zhuofan Ying, Peter Hase, and Mohit Bansal; arXiv 2022 [paper](https://arxiv.org/pdf/2206.11212) [code](https://github.com/zfying/visfis)\n\n- **Identifying Spurious Correlations and Correcting them with an Explanation-based Learning**\n  Misgina Tsighe Hagos, Kathleen Curran, Brian Mac Namee; arXiv 2022 [paper](https://arxiv.org/pdf/2211.08285)\n\n- **Using Explanations to Guide Models**\n  Sukrut Rao, Moritz Böhle, Amin Parchami-Araghi, Bernt Schiele; ICCV 2023 [paper](https://openaccess.thecvf.com/content/ICCV2023/papers/Rao_Studying_How_to_Efficiently_and_Effectively_Guide_Models_with_Explanations_ICCV_2023_paper.pdf) [code](https://github.com/sukrutrao/Model-Guidance)\n\n- **Learning with Explanation Constraints**\n  Rattana Pukdee, Dylan Sam, Zico Kolter, Maria-Florina Balcan, Pradeep Ravikumar; arXiv 2023 [paper](https://arxiv.org/pdf/2303.14496)\n\n- **Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features**\n  Cian Eastwood, Shashank Singh, Andrei Liviu Nicolicioiu, Marin Vlastelica, Julius von Kügelgen, Benrnhard Schölkopf; arXiv 2023 [paper](https://arxiv.org/pdf/2307.09933)\n\n- **Spurious features everywhere-large-scale detection of harmful spurious features in imagenet**\n  Yannic Neuhaus, Maximilian Augustin, Valentyn Boreiko, Matthias Hein; ICCV 2023 [paper](http://openaccess.thecvf.com/content/ICCV2023/papers/Neuhaus_Spurious_Features_Everywhere_-_Large-Scale_Detection_of_Harmful_Spurious_Features_ICCV_2023_paper.pdf) [code](https://github.com/YanNeu/spurious_imagenet)\n\n- **Targeted Activation Penalties Help CNNs Ignore Spurious Signals**\n  Dekai Zhang, Matt Williams, Francesca Toni; AAAI 2024 [paper](https://ojs.aaai.org/index.php/AAAI/article/view/29610) [code](https://github.com/dkaizhang/tap)\n\n----\n\n\n### [Interactive Learning](#content)\n\nApproaches that combine supervision on the explanations with interactive machine learning:\n\n- **Principles of Explanatory Debugging to Personalize Interactive Machine Learning**\n  Todd Kulesza, Margaret Burnett, Weng-Keen Wong, Simone Stumpf; IUI 2015 [paper](https://openaccess.city.ac.uk/id/eprint/13819/1/paper326.pdf)\n\n- **Explanatory Interactive Machine Learning**\n  Stefano Teso, Kristian Kersting; AIES 2019 [paper](https://www.aiml.informatik.tu-darmstadt.de/papers/teso2019aies_XIML.pdf) [code](https://github.com/stefanoteso/caipi)\n  `Notes: introduces explanatory interactive learning, focuses on active learning setup.`\n\n- **Toward Faithful Explanatory Active Learning with Self-explainable Neural Nets**\n  Stefano Teso; IAL Workshop 2019. [paper](https://lirias.kuleuven.be/retrieve/578884) [code](https://github.com/stefanoteso/calimocho)\n  `Notes: explanatory active learning with self-explainable neural networks.`\n\n- **Making deep neural networks right for the right scientific reasons by interacting with their explanations**\n  Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger, Franziska Herbert, Xiaoting Shao, Hans-Georg Luigs, Anne-Katrin Mahlein, Kristian Kersting; Nature Machine Intelligence 2020 [paper](https://www.nature.com/articles/s42256-020-0212-3) [code](https://github.com/ml-research/XIL)\n  `Notes: introduces end-to-end explanatory interactive learning, fixes clever Hans deep neural nets.`\n\n- **Embedding Human Knowledge into Deep Neural Network via Attention Map**\n  Masahiro Mitsuhara, Hiroshi Fukui, Yusuke Sakashita, Takanori Ogata, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi; arXiv 2019 [paper](https://arxiv.org/pdf/1905.03540.pdf)\n\n- **One explanation does not fit all**\n  Kacper Sokol, Peter Flach; 2020 Künstliche Intelligenz [paper](https://link.springer.com/content/pdf/10.1007/s13218-020-00637-y.pdf)\n\n- **FIND: Human-in-the-loop Debugging Deep Text Classifiers**\n  Piyawat Lertvittayakumjorn, Lucia Specia, Francesca Toni; EMNLP 2020 [paper](https://www.aclweb.org/anthology/2020.emnlp-main.24.pdf)\n\n- **Human-driven FOL explanations of deep learning**\n  Gabriele Ciravegna, Francesco Giannini, Marco Gori, Marco Maggini, Stefano Melacci; IJCAI 2020 [paper](https://www.ijcai.org/Proceedings/2020/0309.pdf)\n  `Notes: first-order logic.`\n\n- **Cost-effective Interactive Attention Learning with Neural Attention Process**\n  Jay Heo, Junhyeon Park, Hyewon Jeong, Kwang joon Kim, Juho Lee, Eunho Yang, Sung Ju Hwang; ICML 2020 [paper](https://arxiv.org/pdf/2006.05419.pdf) [code](https://github.com/jayheo/IAL)\n  `Notes: attention, interaction`\n\n- **Soliciting human-in-the-loop user feedback for interactive machine learning reduces user trust and impressions of model accuracy**\n  Donald Honeycutt, Mahsan Nourani, Eric Ragan; AAAI Conference on Human Computation and Crowdsourcing 2020 [paper](https://ojs.aaai.org/index.php/HCOMP/article/download/7464/7291/)\n\n- **ALICE: Active Learning with Contrastive Natural Language Explanations**\n  Weixin Liang, James Zou, Zhou Yu; EMNLP 2020 [paper](https://www.aclweb.org/anthology/2020.emnlp-main.355.pdf)\n\n- **Machine Guides, Human Supervises: Interactive Learning with Global Explanations**\n  Teodora Popordanoska, Mohit Kumar, Stefano Teso; arXiv 2020 [paper](https://arxiv.org/pdf/2009.09723.pdf) [code](https://github.com/tpopordanoska/explanatory-guided-learning)\n  `Notes: introduces narrative bias and explanatory guided learning, focuses on human-initiated interaction and global explanations.`\n\n- **Teaching an Active Learner with Contrastive Examples**\n  Chaoqi Wang, Adish Singla, Yuxin Chen. NeurIPS 2021. [paper](https://proceedings.neurips.cc/paper/2021/file/958adb57686c2fdec5796398de5f317a-Paper.pdf)\n\n- **Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations**\n  Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting; CVPR 2021 [paper](https://arxiv.org/pdf/2011.12854) [code](https://github.com/ml-research/NeSyXIL)\n  `Notes: first-order logic, attention.`\n\n- **Right for Better Reasons: Training Differentiable Models by Constraining their Influence Function**\n  Xiaoting Shao, Arseny Skryagin, Patrick Schramowski, Wolfgang Stammer, Kristian Kersting; AAAI 2021 [paper](https://www.aaai.org/AAAI21Papers/AAAI-5436.ShaoX.pdf)\n\n- **User Driven Model Adjustment via Boolean Rule Explanations**\n  Elizabeth Daly, Massimiliano Mattetti, Öznur Alkan, Rahul Nair; AAAI 2021 [paper](https://ojs.aaai.org/index.php/AAAI/article/view/16737/16544)\n\n- **Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers**\n  Bhavya Ghai, Vera Liao, Yunfeng Zhang, Rachel Bellamy, Klaus Mueller. Proc. ACM Hum.-Comput. Interact. 2021 [paper](https://www3.cs.stonybrook.edu/~mueller/papers/CSCW%20XAL%202020.pdf)\n\n- **Bandits for Learning to Explain from Explanations**\n  Freya Behrens, Stefano Teso, Davide Mottin; XAI Workshop 2021 [paper](https://arxiv.org/pdf/2102.03815) [code](https://github.com/stefanoteso/explearner-simpler)\n  `Notes: preliminary.`\n\n- **HILDIF: Interactive Debugging of NLI Models Using Influence Functions**\n  Hugo Zylberajch, Piyawat Lertvittayakumjorn, Francesca Toni; InterNLP Workshop 2021 [paper](https://aclanthology.org/2021.internlp-1.1.pdf) [code](https://github.com/hugozylberajch/HILDIF)\n\n- **Refining Neural Networks with Compositional Explanations**\n  Huihan Yao, Ying Chen, Qinyuan Ye, Xisen Jin, Xiang Ren; arXiv 2021 [paper](https://arxiv.org/pdf/2103.10415) [code](https://github.com/INK-USC/expl-refinement)\n\n- **Interactive Label Cleaning with Example-based Explanations**\n  Stefano Teso, Andrea Bontempelli, Fausto Giunchiglia, Andrea Passerini; NeurIPS 2021 [paper](https://arxiv.org/pdf/2106.03922) [code](https://github.com/abonte/cincer)\n\n- **Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systems**\n  Subbarao Kambhampati, Sarath Sreedharan, Mudit Verma, Yantian Zha, Lin Guan; AAAI 2022 [paper](https://arxiv.org/pdf/2109.09904)\n\n- **Toward a Unified Framework for Debugging Gray-box Models**\n  Andrea Bontempelli, Fausto Giunchiglia, Andrea Passerini, Stefano Teso; AAAI-22 Workshop on Interactive Machine Learning [paper](https://arxiv.org/pdf/2109.11160)\n\n- **Active Learning by Acquiring Contrastive Examples**\n  Katerina Margatina, Giorgos Vernikos, Loïc Barrault, Nikolaos Aletras; EMNLP 2021 [paper](https://aclanthology.org/2021.emnlp-main.51.pdf) [code](https://github.com/mourga/contrastive-active-learning)\n\n- **Finding and Fixing Spurious Patterns with Explanations**\n  Gregory Plumb, Marco Tulio Ribeiro, Ameet Talwalkar; arXiv 2021 [paper](https://arxiv.org/pdf/2106.02112)\n\n- **Interactively Generating Explanations for Transformer Language Models**\n  Patrick Schramowski, Felix Friedrich, Christopher Tauchmann, Kristian Kersting; arXiv 2021 [paper](https://arxiv.org/pdf/2110.02058.pdf)\n\n- **Interaction with Explanations in the XAINES Project**\n  Mareike Hartmann, Ivana Kruijff-Korbayová, Daniel Sonntag; arXiv 2021 [paper](https://dataninja.nrw/wp-content/uploads/2021/09/7_Hartmann_XAINES_Abstract.pdf)\n\n- **A Rationale-Centric Framework for Human-in-the-loop Machine Learning**\n  Jinghui Lu, Linyi Yang, Brian Mac Namee, Yue Zhang; ACL 2022 [paper](https://aclanthology.org/2022.acl-long.481.pdf) [code](https://github.com/GeorgeLuImmortal/RDL-Rationales-centric-Double-robustness-Learning/)\n\n- **A Typology to Explore and Guide Explanatory Interactive Machine Learning**\n  Felix Friedrich, Wolfgang Stammer, Patrick Schramowski, Kristian Kersting; arXiv 2022 [paper](https://arxiv.org/pdf/2203.03668)\n\n- **CAIPI in Practice: Towards Explainable Interactive Medical Image Classification**\n  Emanuel Slany, Yannik Ott, Stephan Scheele, Jan Paulus, Ute Schmid; IFIP International Conference on Artificial Intelligence Applications and Innovations, 2022 [paper](https://arxiv.org/pdf/2204.02661)\n\n- **Semantic Interactive Learning for Text Classification: A Constructive Approach for Contextual Interactions**\n  Semastian Kiefer, Mareike Hoffmann, Ute Schmid; Machine Learning and Knowledge Extraction, 2022 [paper](https://www.mdpi.com/2504-4990/4/4/50/pdf)\n\n- **Impact of Feedback Type on Explanatory Interactive Learning**\n  Misgina Tsighe Hagos, Kathleen Curran, Brian Mac Namee; ISMIS 2022 [paper](https://arxiv.org/pdf/2209.12476)\n\n- **Leveraging Explanations in Interactive Machine Learning: An Overview**\n  Stefano Teso, Öznur Alkan, Wolfgang Stammer, Elizabeth Daly; Frontiers in AI 2023 [paper](https://www.frontiersin.org/articles/10.3389/frai.2023.1066049/full) [preprint](https://arxiv.org/pdf/2207.14526)\n\n- **Concept-level Debugging of Part-prototype Networks**\n  Andrea Bontempelli, Stefano Teso, Fausto Giunchiglia, Andrea Passerini; ICLR 2023 [paper](https://openreview.net/pdf?id=oiwXWPDTyNk) [code](https://github.com/abonte/protopdebug)\n\n- **Learning to Intervene on Concept Bottlenecks**\n  David Steinmann, Wolfgang Stammer, Felix Friedrich, Kristian Kersting; arXiv 2023 [paper](https://arxiv.org/pdf/2308.13453)\n\n----\n\n\n### [Reinforcement Learning](#content)\n\n- **Explanation Augmented Feedback in Human-in-the-Loop Reinforcement Learning**\n  Lin Guan, Mudit Verma, Sihang Guo, Ruohan Zhang, Subbarao Kambhampati; Human And Machine in-the-Loop Evaluation and Learning Strategies [paper](https://arxiv.org/pdf/2006.14804)\n\n- **Learning from explanations and demonstrations: A pilot study**\n  Silvia Tulli, Sebastian Wallkötter, Ana Paiva, Francisco Melo, Mohamed Chetouani; Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence 2020 [paper](https://aclanthology.org/2020.nl4xai-1.13.pdf)\n\n- **Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation**\n  Lin Guan, Mudit Verma, Sihang Guo, Ruohan Zhang, Subbarao Kambhampati; NeurIPS 2021 [pdf](https://arxiv.org/pdf/2006.14804.pdf)\n\n----\n\n\n### [Distillation](#content)\n\n- **Model reconstruction from model explanations**\n  Smitha Milli, Ludwig Schmidt, Anca D. Dragan, Moritz Hardt; FAcct 2019 [paper](https://arxiv.org/pdf/1807.05185)\n\n- **Evaluating Explanations: How much do explanations from the teacher aid students?**\n  Danish Pruthi, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C. Lipton, Graham Neubig, and William W. Cohen; arXiv 2020 [paper](https://arxiv.org/pdf/2012.00893)\n  `Notes: defines importance of different kinds of explanations by measuring their impact when used as supervision.`\n\n----\n\n\n### [Regularization without Supervision](#content)\n\nApproaches that regularize the model's explanations in an unsupervised manner, often for improved interpretability.\n\n- **Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients**\n  Andrew Ross and Finale Doshi-Velez. AAAI 2018 [paper](https://ojs.aaai.org/index.php/AAAI/article/view/11504/11363)\n\n- **Towards robust interpretability with self-explaining neural networks**\n  David Alvarez-Melis, Tommi Jaakkola; NeurIPS 2018 [paper](https://proceedings.neurips.cc/paper/2018/file/3e9f0fc9b2f89e043bc6233994dfcf76-Paper.pdf)\n\n- **Beyond sparsity: Tree regularization of deep models for interpretability**\n  Mike Wu, Michael Hughes, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, Finale Doshi-Velez; AAAI 2018 [paper](https://ojs.aaai.org/index.php/AAAI/article/view/11501/11360)\n\n- **Regional tree regularization for interpretability in deep neural networks**\n  Mike Wu, Sonali Parbhoo, Michael Hughes, Ryan Kindle, Leo Celi, Maurizio Zazzi, Volker Roth, Finale Doshi-Velez; AAAI 2020 [paper](https://ojs.aaai.org/index.php/AAAI/article/view/6112/5968)\n\n- **Regularizing black-box models for improved interpretability**\n  Gregory Plumb, Maruan Al-Shedivat, Ángel Alexander Cabrera, Adam Perer, Eric Xing, Ameet Talwalkar; NeurIPS 2020 [paper](https://papers.nips.cc/paper/2020/file/770f8e448d07586afbf77bb59f698587-Paper.pdf)\n\n- **Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias**\n  Krishna Kumar Singh, Dhruv Mahajan, Kristen Grauman, Yong Jae Lee, Matt Feiszli, Deepti Ghadiyaram; CVPR 2020 [paper](http://openaccess.thecvf.com/content_CVPR_2020/papers/Singh_Dont_Judge_an_Object_by_Its_Context_Learning_to_Overcome_CVPR_2020_paper.pdf) [code](https://github.com/princetonvisualai/ContextualBias)\n\n- **Trustworthy convolutional neural networks: A gradient penalized-based approach**\n  Nicholas Halliwell, Freddy Lecue; arXiv 2020 [paper](https://arxiv.org/pdf/2009.14260)\n\n- **Explainable Models with Consistent Interpretations**\n  Vipin Pillai, Hamed Pirsiavash; AAAI 2021 [paper](https://www.csee.umbc.edu/~hpirsiav/papers/gc_aaai21.pdf) [code](https://github.com/UMBCvision/Explainable-Models-with-Consistent-Interpretations)\n\n- **Explanation Consistency Training: Facilitating Consistency-based Semi-supervised Learning with Interpretability**\n  Tao Han, Wei-Wei Tu, Yu-Feng Li; AAAI 2021 [paper](https://www.aaai.org/AAAI21Papers/AAAI-7186.HanT.pdf)\n\n- **Improving Deep Learning Interpretability by Saliency Guided Training**\n  Aya Abdelsalam Ismail, Hector Corrada Bravo, Soheil Feizi; NeurIPS 2021 [paper](https://proceedings.neurips.cc//paper/2021/file/e0cd3f16f9e883ca91c2a4c24f47b3d9-Paper.pdf) [code](https://github.com/ayaabdelsalam91/saliency_guided_training)\n\n- **Generating Deep Networks Explanations with Robust Attribution Alignment**\n  Guohang Zeng, Yousef Kowsar, Sarah Erfani, James Bailey; ACML 2021 [paper](https://proceedings.mlr.press/v157/zeng21b/zeng21b.pdf)\n\n- **Learning by Self-Explaining**\n  Wolfgang Stammer, Felix Friedrich, David Steinmann, Hikaru Shindo, Kristian Kersting; arXiv 2023 [paper](https://arxiv.org/pdf/2309.08395)\n\n- **Spurious Correlations in Concept Drift: Can Explanatory Interaction Help?**\n  Cristiana Lalletti, Stefano Teso; arXiv 2024 [paper](https://arxiv.org/pdf/2407.16515)\n\n\n----\n\n\n### [Machine Teaching](#content)\n\n- **Interpretable Machine Teaching via Feature Feedback**\n  Shihan Su, Yuxin Chen, Oisin Mac Aodha, Pietro Perona, Yisong Yue; Workshop on Teaching Machines, Robots, and Humans 2017 [paper](https://authors.library.caltech.edu/87329/1/nips17-teaching_paper-5.pdf)\n\n- **Teaching Categories to Human Learners with Visual Explanations**\n  Oisin Mac Aodha, Shihan Su, Yuxin Chen, Pietro Perona, Yisong Yue; CVPR 2018 [paper](https://openaccess.thecvf.com/content_cvpr_2018/papers/Aodha_Teaching_Categories_to_CVPR_2018_paper.pdf)\n  `Notes: this is *inverse* teaching, i.e., machine teaches human.`\n\n----\n\n### [Applications](#applications)\n\n- **Improving a neural network model by explanation-guided training for glioma classification based on MRI data**\n  Frantisek Sefcik, Wanda Benesova; arXiv 2021 [paper](https://arxiv.org/pdf/2107.02008)\n  `Notes: based on layer-wise relevance propagation.`\n\n---\n\n\n### [Related Works](#content)\n\nExplanation-based learning, focuses on logic-based formalisms and learning strategies:\n\n- **Explanation-based generalization: A unifying view**\n  Tom Mitchell, Richard Keller, Smadar Kedar-Cabelli; MLJ 1986 [paper](https://link.springer.com/content/pdf/10.1023/A:1022691120807.pdf)\n\n- **Explanation-based learning: An alternative view**\n  Gerald DeJong, Raymond Mooney; MLJ 1986 [paper](https://link.springer.com/content/pdf/10.1007/BF00114116.pdf)\n\n- **Explanation-based learning: A survey of programs and perspectives**\n  Thomas Ellman; ACM Computing Surveys 1989 [paper](https://academiccommons.columbia.edu/doi/10.7916/D8SF343S/download)\n\n- **Probabilistic explanation based learning**\n  Angelika Kimmig, Luc De Raedt, Hannu Toivonen; ECML 2007 [paper](https://link.springer.com/content/pdf/10.1007/978-3-540-74958-5_19.pdf)\n\n\nInjecting invariances / feature constraints into models:\n\n- **Tangent Prop - A formalism for specifying selected invariances in an adaptive network**\n  Patrice Simard, Bernard Victorri, Yann Le Cun, John Denker; NeurIPS 1992 [paper](https://papers.nips.cc/paper/536-tangent-prop-a-formalism-for-specifying-selected-invariances-in-an-adaptive-network.pdf)\n  `Notes: injects invariances into a neural net by regularizing its gradient; precursor to learning from gradient-based explanations.`\n\n- **Training invariant support vector machines**\n  Dennis DeCoste, Bernhard Schölkopf; MLJ 2002 [paper](https://link.springer.com/content/pdf/10.1023/A:1012454411458.pdf)\n\n- **The constrained weight space svm: learning with ranked features**\n  Kevin Small, Byron Wallace, Carla Brodley, Thomas Trikalinos; ICML 2011 [paper](http://www.icml-2011.org/papers/465_icmlpaper.pdf)\n\n\nDual label-feature feedback:\n\n- **Active learning with feedback on features and instances**\n  Hema Raghavan, Omid Madani, Rosie Jones; JMLR 2006 [paper](https://www.jmlr.org/papers/volume7/raghavan06a/raghavan06a.pdf)\n\n- **An interactive algorithm for asking and incorporating feature feedback into support vector machines**\n  Hema Raghavan, James Allan; ACM SIGIR 2007 [paper](https://www.academia.edu/download/49512889/IR-560.pdf)\n\n- **Learning from labeled features using generalized expectation criteria**\n  Gregory Druck, Gideon Mann, Andrew McCallum;  ACM SIGIR 2008 [paper](http://www.cs.umass.edu/~mccallum/papers/druck08sigir.pdf)\n\n- **Active learning by labeling features**\n  Gregory Druck, Burr Settles, Andrew McCallum; EMNLP 2009 [paper](https://www.aclweb.org/anthology/D09-1009.pdf)\n\n- **A unified approach to active dual supervision for labeling features and examples**\n  Josh Attenberg, Prem Melville, Foster Provost; ECML-PKDD 2010 [paper](https://link.springer.com/content/pdf/10.1007/978-3-642-15880-3_9.pdf)\n\n- **Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances**\n  Burr Settles; EMNLP 2011 [paper](https://www.aclweb.org/anthology/D11-1136.pdf)\n\n- **Learning from discriminative feature feedback**\n  Sanjoy Dasgupta, Akansha Dey, Nicholas Roberts, Sivan Sabato; NeurIPS 2018 [paper](http://papers.neurips.cc/paper/7651-learning-from-discriminative-feature-feedback.pdf)\n\n- **Robust Learning from Discriminative Feature Feedback**\n  Sanjoy Dasgupta, Sivan Sabato; AISTATS 2020 [paper](http://proceedings.mlr.press/v108/dasgupta20a/dasgupta20a-supp.pdf)\n\n- **Practical Benefits of Feature Feedback Under Distribution Shift**\n  Anurag Katakkar, Weiqin Wang, Clay Yoo, Zachary Lipton, Divyansh Kaushik; arXiv 2021 [paper](https://arxiv.org/pdf/2110.07566)\n\n\nLearning from rationales:\n\n- **Using “annotator rationales” to improve machine learning for text categorization**\n  Omar Zaidan, Jason Eisner, Christine Piatko; NAACL 2007 [paper](https://www.aclweb.org/anthology/N07-1033.pdf)\n\n- **Modeling annotators: A generative approach to learning from annotator rationales**\n  Omar Zaidan, Jason Eisner; EMNLP 2008 [paper](https://www.aclweb.org/anthology/D08-1004.pdf)\n\n- **Active learning with rationales for text classification**\n  Manali Sharma, Di Zhuang, Mustafa Bilgic; NAACL 2015 [paper](https://www.aclweb.org/anthology/N15-1047.pdf)\n\n\nCounterfactual augmentation:\n\n- **Learning The Difference That Makes A Difference With Counterfactually-Augmented Data**\n  Divyansh Kaushik, Eduard Hovy, Zachary Lipton; ICLR 2019 [paper](https://openreview.net/pdf?id=Sklgs0NFvr) [code](https://github.com/dkaushik96/counterfactually-augmented-data)\n\n- **Explaining the Efficacy of Counterfactually Augmented Data**\n  Divyansh Kaushik, Amrith Setlur, Eduard H. Hovy, Zachary Lipton; ICLR 2021. [paper](https://openreview.net/pdf?id=HHiiQKWsOcV) [code](https://github.com/acmi-lab/counterfactually-augmented-data)\n\n- **An Investigation of the (In)effectiveness of Counterfactually-augmented Data**\n  Nitish Joshi, He He; arXiv 2021 [paper](https://arxiv.org/pdf/2107.00753)\n\n\nCritiquing in recommenders:\n\n- **Critiquing-based recommenders: survey and emerging trends**\n  Li Chen, Pearl Pu; User Modeling and User-Adapted Interaction 2012 [paper](https://link.springer.com/content/pdf/10.1007/s11257-011-9108-6.pdf)\n\n- **Coactive critiquing: Elicitation of preferences and features**\n  Stefano Teso, Paolo Dragone, Andrea Passerini; AAAI 2017 [paper](https://ojs.aaai.org/index.php/AAAI/article/view/10929/10788)\n\n\nGray-box models:\n\n- **Concept bottleneck models**\n  Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang; ICML 2020 [paper](http://proceedings.mlr.press/v119/koh20a/koh20a.pdf)\n\n----\n\n\n### [Resources](#content)\n\nA selection of general resources on Explainable AI focusing on overviews, surveys, societal implications, and critiques:\n\n- **Survey and critique of techniques for extracting rules from trained artificial neural networks**\n  Robert Andrews, Joachim Diederich, Alan B. Tickle; Knowledge-based systems 1995 [page](https://www.sciencedirect.com/science/article/abs/pii/0950705196819204)\n\n- **Toward harnessing user feedback for machine learning**\n  Simone Stumpf, Vidya Rajaram, Lida Li, Margaret Burnett, Thomas Dietterich, Erin Sullivan, Russell Drummond, Jonathan Herlocker; IUI 2007 [paper](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.140\u0026rep=rep1\u0026type=pdf)\n\n- **The Mythos of Model Interpretability**\n  Zachary Lipton; CACM 2016 [paper](https://dl.acm.org/doi/pdf/10.1145/3236386.3241340)\n\n- **A survey of methods for explaining black box models**\n  Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi; ACM Computing Surveys 2018 [paper](https://dl.acm.org/doi/pdf/10.1145/3236009)\n\n- **Sanity checks for saliency maps**\n  Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim; NeurIPS 2018 [paper](http://papers.neurips.cc/paper/8160-sanity-checks-for-saliency-maps.pdf) [code](https://github.com/adebayoj/sanity_checks_saliency)\n\n- **Recognition in terra incognita**\n  Sara Beery, Grant Van Horn, Pietro Perona; ECCV 2018 [paper](https://openaccess.thecvf.com/content_ECCV_2018/papers/Beery_Recognition_in_Terra_ECCV_2018_paper.pdf)\n\n- **Explanation in Artificial Intelligence: Insights from the Social Sciences**\n  Tim Miller; Artificial Intelligence, 2019 [paper](https://www.sciencedirect.com/science/article/abs/pii/S0004370218305988)\n\n- **Unmasking clever hans predictors and assessing what machines really learn**\n  Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojciech Samek, Klaus-Robert Müller; Nature Communications 2019 [paper](https://www.nature.com/articles/s41467-019-08987-4)\n\n- **Interpretation of neural networks is fragile**\n  Amirata Ghorbani, Abubakar Abid, James Zou; AAAI 2019 [paper](https://ojs.aaai.org/index.php/AAAI/article/view/4252/4130)\n\n- **A Benchmark for Interpretability Methods in Deep Neural Networks**\n  Sara Hooke, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim; NeurIPS 2019 [paper](http://papers.neurips.cc/paper/9167-a-benchmark-for-interpretability-methods-in-deep-neural-networks.pdf) [code](https://github.com/google-research/google-research/tree/master/interpretability_benchmark)\n\n- **Is Attention Interpretable?**\n  Sofia Serrano, Noah A. Smith; ACL 2019 [paper](https://www.aclweb.org/anthology/P19-1282.pdf)\n\n- **Attention is not Explanation**\n  Sarthak Jain, and Byron C. Wallace; ACL 2019 [paper](https://www.aclweb.org/anthology/N19-1357.pdf)\n\n- **Attention is not not Explanation**\n  Sarah Wiegreffe, and Yuval Pinter; EMNLP-IJCNLP 2019 [paper](https://www.aclweb.org/anthology/D19-1002.pdf)\n\n- **The (un)reliability of saliency methods**\n  Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, and Been Kim; Explainable AI: Interpreting, Explaining and Visualizing Deep Learning 2019 [paper](http://interpretable-ml.org/nips2017workshop/papers/10.pdf)\n\n- **Explanations can be manipulated and geometry is to blame**\n  Ann-Kathrin Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus-Robert Müller, and Pan Kessel; NeurIPS 2019 [paper](https://papers.nips.cc/paper/2019/file/bb836c01cdc9120a9c984c525e4b1a4a-Paper.pdf)\n\n- **Fooling Neural Network Interpretations via Adversarial Model Manipulation**\n  Juyeon Heo, Sunghwan Joo, and Taesup Moon; NeurIPS 2019 [paper](https://proceedings.neurips.cc/paper/2019/file/7fea637fd6d02b8f0adf6f7dc36aed93-Paper.pdf)\n\n- **Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead**\n  Cynthia Rudin; Nature Machine Intelligence 2019 [page](https://www.nature.com/articles/s42256-019-0048-x)\n\n- **The Principles and Limits of Algorithm-in-the-loop Decision Making**\n  Ben Green, Yiling Chen; PACM HCI 2019 [paper](https://dl.acm.org/doi/pdf/10.1145/3359152)\n\n- **Shortcut learning in deep neural networks**\n  Robert Geirhos, Jorn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, Felix Wichmann; Nature Machine Intelligence 2020 [page](https://www.nature.com/articles/s42256-020-00257-z)\n\n- **When Explanations Lie: Why Many Modified BP Attributions Fail**\n  Leon Sixt, Maximilian Granz, Tim Landgraf. ICML 2020 [paper](http://proceedings.mlr.press/v119/sixt20a/sixt20a.pdf)\n\n- **The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?**\n  Jasmijn Bastings, Katja Filippova; Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP 2020 [paper](https://aclanthology.org/2020.blackboxnlp-1.14.pdf)\n\n- **Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models**\n  Christopher Grimsley, Elijah Mayfield, Julia Bursten;  Language Resources and Evaluation Conference 2020 [paper](https://aclanthology.org/2020.lrec-1.220.pdf)\n\n- **AI for radiographic COVID-19 detection selects shortcuts over signal**\n  Alex DeGrave, Joseph Janizek, Su-In Lee; Nature Machine Intelligence 2021 [paper](https://www.nature.com/articles/s42256-021-00338-7.pdf) [code](https://github.com/suinleelab/cxr_covid)\n\n- **How Well do Feature Visualizations Support Causal Understanding of CNN Activations?**\n  Roland Zimmermann,Judy Borowski, Robert Geirhos, Matthias Bethge, Thomas Wallis, Wieland Brendel; arXiv 2021 [paper](https://arxiv.org/pdf/2106.12447)\n\n- **Post hoc explanations may be ineffective for detecting unknown spurious correlation**\n  Julius Adebayo, Michael Muelly, Harold Abelson, and Been Kim; ICLR 2022 [paper](https://openreview.net/pdf?id=xNOVfCCvDpM) [code](https://github.com/adebayoj/posthocspurious)\n\n- **Where is the Truth? The Risk of Getting Confounded in a Continual World**\n  Florian Peter Busch, Roshni Kamath, Rupert Mitchell, Wolfgang Stammer, Kristian Kersting, Martin Mundt; arXiv 2024 [paper](https://arxiv.org/pdf/2402.06434)\n\n- **Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation**\n  David Steinmann, Felix Divo, Maurice Kraus, Antonia Wüst, Lukas Struppek, Felix Friedrich, and Kristian Kersting; arXiv 2024 [paper](https://arxiv.org/pdf/2412.05152?)\n\n----\n\n\n### Related Lists\n\n- [Awesome explainable AI](https://github.com/wangyongjie-ntu/Awesome-explainable-AI)\n\n- [Awesome machine learning interpretability](https://github.com/jphall663/awesome-machine-learning-interpretability)\n\n----\n\n\n### Not Yet Sorted\n\n- **Multimodal explanations: Justifying decisions and pointing to the evidence**\n  Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, Marcus Rohrbach; CVPR 2018 [paper](https://openaccess.thecvf.com/content_cvpr_2018/papers/Park_Multimodal_Explanations_Justifying_CVPR_2018_paper.pdf)\n\n- **Learning Deep Attribution Priors Based On Prior Knowledge**\n  Ethan Weinberger, Joseph Janizek, Su-In Lee; NeurIPS 2020 [paper](https://papers.nips.cc/paper/2020/file/a19883fca95d0e5ec7ee6c94c6c32028-Paper.pdf)\n\n----\n\n\n### TODO\n\n- Crawl \u0026 reference work on NLP.\n\n\n\n### Comments\n\nThis list is directly inspired by all the awesome awesome lists out there!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanoteso%2Fawesome-explanatory-supervision","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstefanoteso%2Fawesome-explanatory-supervision","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanoteso%2Fawesome-explanatory-supervision/lists"}