{"id":18524963,"url":"https://github.com/paddlepaddle/interpretdl","last_synced_at":"2025-04-06T01:05:54.782Z","repository":{"id":41163910,"uuid":"274012656","full_name":"PaddlePaddle/InterpretDL","owner":"PaddlePaddle","description":"InterpretDL: Interpretation of Deep Learning Models，基于『飞桨』的模型可解释性算法库。","archived":false,"fork":false,"pushed_at":"2023-10-14T09:33:07.000Z","size":33468,"stargazers_count":231,"open_issues_count":4,"forks_count":36,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-05-14T12:02:10.470Z","etag":null,"topics":["convolutional-neural-networks","explanations","grad-cam","interpretation-algorithms","lime","model-interpretation","nlp-models","paddlepaddle","smoothgrad","vision-transformer","visualizations"],"latest_commit_sha":null,"homepage":"https://interpretdl.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PaddlePaddle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-06-22T01:26:45.000Z","updated_at":"2024-05-12T05:04:57.000Z","dependencies_parsed_at":"2023-10-15T09:43:02.768Z","dependency_job_id":"45dd083b-9463-4260-b6de-fac7bd262c70","html_url":"https://github.com/PaddlePaddle/InterpretDL","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PaddlePaddle%2FInterpretDL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PaddlePaddle%2FInterpretDL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PaddlePaddle%2FInterpretDL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PaddlePaddle%2FInterpretDL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PaddlePaddle","download_url":"https://codeload.github.com/PaddlePaddle/InterpretDL/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247419859,"owners_count":20936012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","explanations","grad-cam","interpretation-algorithms","lime","model-interpretation","nlp-models","paddlepaddle","smoothgrad","vision-transformer","visualizations"],"created_at":"2024-11-06T17:44:01.812Z","updated_at":"2025-04-06T01:05:54.752Z","avatar_url":"https://github.com/PaddlePaddle.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[**中文**](./README_CN.md) | English\n\n[![Release](https://img.shields.io/github/release/PaddlePaddle/InterpretDL.svg)](https://github.com/PaddlePaddle/InterpretDL/releases)\n[![PyPI](https://img.shields.io/pypi/v/interpretdl.svg)](https://pypi.org/project/interpretdl)\n[![CircleCI](https://circleci.com/gh/PaddlePaddle/InterpretDL.svg?style=shield)](https://circleci.com/gh/PaddlePaddle/InterpretDL)\n[![Documentation Status](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](https://interpretdl.readthedocs.io/en/latest/index.html)\n[![Downloads](https://static.pepy.tech/personalized-badge/interpretdl?period=total\u0026units=abbreviation\u0026left_color=grey\u0026right_color=green\u0026left_text=Downloads%20Total)](https://pepy.tech/project/interpretdl)\n\n![logo](https://user-images.githubusercontent.com/13829174/192076322-72b1306b-58c9-4d34-89a1-28d168ebf079.png)\n\n# InterpretDL: Interpretation of Deep Learning Models based on PaddlePaddle\n\nInterpretDL, short for *interpretations of deep learning models*, is a model interpretation toolkit for [PaddlePaddle](https://github.com/PaddlePaddle/Paddle) models. This toolkit contains implementations of many interpretation algorithms, including LIME, Grad-CAM, Integrated Gradients and more. Some SOTA and new interpretation algorithms are also implemented.\n\n*InterpretDL is under active construction and all contributions are welcome!*\n\n![](https://user-images.githubusercontent.com/13829174/159609890-bf3f2050-1ee8-482f-baac-693d280f9039.jpg)\n\n# Why InterpretDL\n\nThe increasingly complicated deep learning models make it impossible for people to understand their internal workings. Interpretability of black-box models has become the research focus of many talented researchers. InterpretDL provides a collection of both classical and new algorithms for interpreting models.\n\nBy utilizing these helpful methods, people can better understand why models work and why they don't, thus contributing to the model development process.\n\nFor researchers working on designing new interpretation algorithms, InterpretDL gives an easy access to existing methods that they can compare their work with.\n\n# :fire: :fire: :fire: News :fire: :fire: :fire:\n\n- (2024.6) A paper investigating [feature attributions of LLMs via Optimal Transport](https://openreview.net/forum?id=qKL25sGjxL) got accepted by ICML'24. See implementations at [GiLOT](https://github.com/holyseven/GiLOT).\n\n\u003e Xuhong Li, Jiamin Chen, Yekun Chai, Haoyi Xiong. GILOT: Interpreting Generative Language Models via Optimal Transport. ICML 2024. [paper link](https://openreview.net/pdf?id=qKL25sGjxL).\n\n- (2023.10) M4 XAI Benchmark got accepted by Neurips'23 Datasets and Benchmarks. See implementations at [M4_XAI_Benchmark](https://github.com/holyseven/M4_XAI_Benchmark).\n\n\u003e Xuhong Li, Mengnan Du, Jiamin Chen, Yekun Chai, Himabindu Lakkaraju, Haoyi Xiong. “M4: A Unified XAI Benchmark for Faithfulness Evaluation of Feature Attribution Methods across Metrics, Modalities and Models.” Neurips 2023, Dataset and Benchmark Track. [paper link](https://openreview.net/forum?id=6zcfrSz98y).\n\n- (2023.2) One paper on explaining Transformers got accepted by TMLR. See implementations at [BT](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/bidirectional_transformer.py).\n\n\u003e Jiamin Chen, Xuhong Li, Lei Yu, Dejing Dou, Haoyi Xiong. “Beyond Intuition: Rethinking Token Attributions inside Transformers.” TMLR. [paper link](https://openreview.net/forum?id=rm0zIzlhcX).\n\n# Demo\n\nInterpretation algorithms give a hint of why a black-box model makes its decision.\n\nThe following table gives visualizations of several interpretation algorithms applied to the original image to tell us why the model predicts \"bull_mastiff.\"\n\n|    Original Image    | IntGrad ([demo](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/example_int_grad_cv.ipynb)) | SG ([demo](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/example_smooth_grad_cv.ipynb)) | LIME ([demo](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/example_lime_cv.ipynb)) | Grad-CAM ([demo](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/example_grad_cam_cv.ipynb)) |\n| :------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |\n| ![](imgs/catdog.jpg) |               ![](imgs/catdog_ig_overlay.jpeg)               |               ![](imgs/catdog_sg_overlay.jpeg)               |              ![](imgs/catdog_lime_overlay.jpeg)              |            ![](imgs/catdog_gradcam_overlay.jpeg)             |\n\nFor sentiment analysis task, the reason why a model gives positive/negative predictions can be visualized as follows. A quick demo can be found [here](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/ernie-2.0-en-sst-2.ipynb). Samples in Chinese are also available [here](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/ernie-1.0-zh-chnsenticorp.ipynb).\n\n![](imgs/sentiment-en.png)\n\n\n# Contents\n\n- [InterpretDL: Interpretation of Deep Learning Models based on PaddlePaddle](#interpretdl-interpretation-of-deep-learning-models-based-on-paddlepaddle)\n- [Why InterpretDL](#why-interpretdl)\n- [:fire: :fire: :fire: News :fire: :fire: :fire:](#fire-fire-fire-news-fire-fire-fire)\n- [Demo](#demo)\n- [Contents](#contents)\n- [Installation](#installation)\n  - [Pip installation](#pip-installation)\n  - [Developer installation](#developer-installation)\n  - [Unit Tests](#unit-tests)\n- [Documentation](#documentation)\n- [Getting Started](#getting-started)\n- [Examples and Tutorials](#examples-and-tutorials)\n- [Roadmap](#roadmap)\n  - [Implemented Algorithms with Taxonomy](#implemented-algorithms-with-taxonomy)\n  - [Implemented Trustworthiness Evaluation Algorithms](#implemented-trustworthiness-evaluation-algorithms)\n- [Presentations](#presentations)\n- [References of Algorithms](#references-of-algorithms)\n- [Copyright and License](#copyright-and-license)\n- [Recent News](#recent-news)\n\n# Installation\n\nIt requires the deep learning framework [paddlepaddle](https://www.paddlepaddle.org.cn/install/quick), versions with CUDA support are recommended.\n\n## Pip installation\n\n```bash\npip install interpretdl\n\n# or with tsinghua mirror\npip install interpretdl -i https://pypi.tuna.tsinghua.edu.cn/simple\n```\n\n## Developer installation\n\n```bash\ngit clone https://github.com/PaddlePaddle/InterpretDL.git\n# ... fix bugs or add new features\ncd InterpretDL \u0026\u0026 pip install -e .\n# welcome to propose pull request and contribute\nyapf -i \u003cpython_file_path\u003e  # code style: column_limit=120\n```\n\n## Unit Tests\n\n```bash\n# run gradcam unit tests\npython -m unittest -v tests.interpreter.test_gradcam\n# run all unit tests\npython -m unittest -v\n```\n\n# Documentation\n\nOnline link: [interpretdl.readthedocs.io](https://interpretdl.readthedocs.io/en/latest/index.html).\n\nOr generate the docs locally:\n\n```bash\ngit clone https://github.com/PaddlePaddle/InterpretDL.git\ncd docs\nmake html\nopen _build/html/index.html\n```\n\n# Getting Started\n\nAll interpreters inherit the abstract class [`Interpreter`](https://github.com/PaddlePaddle/InterpretDL/blob/4f7444160981e99478c26e2a52f8e40bd06bf644/interpretdl/interpreter/abc_interpreter.py), of which `interpret(**kwargs)` is the function to call.\n\n```python\n# an example of SmoothGradient Interpreter.\n\nimport interpretdl as it\nfrom paddle.vision.models import resnet50\npaddle_model = resnet50(pretrained=True)\nsg = it.SmoothGradInterpreter(paddle_model, use_cuda=True)\ngradients = sg.interpret(\"test.jpg\", visual=True, save_path=None)\n```\n\nA quick [Getting-Started tutorial](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/Getting_Started.ipynb) (or [on NBviewer](https://nbviewer.org/github/PaddlePaddle/InterpretDL/blob/master/tutorials/Getting_Started.ipynb)) is provided. It takes only a few minutes to be familiar with InterpretDL.\n\n# Examples and Tutorials\n\nWe have provided at least one example for each interpretation algorithm and each trustworthiness evaluation algorithm, hopefully covering applications for both CV and NLP.\n\nWe are currently preparing tutorials for easy usages of InterpretDL.\n\nBoth examples and tutorials can be accessed under [tutorials](https://github.com/PaddlePaddle/InterpretDL/tree/master/tutorials) folder.\n\n# Roadmap\n\nWe are planning to create a useful toolkit for offering the model interpretations as well as evaluations.\nWe have now implemented the interpretation algorithms as follows, and we are planning to add more algorithms that are desired.\nWelcome to contribute or just tell us which algorithms are desired.\n\n## Implemented Algorithms with Taxonomy\n\nTwo dimensions (representations of explanation results and types of the target model) are used to categorize the interpretation algorithms. This taxonomy can be an indicator to find the best suitable algorithm for the target task and model.\n\n| Methods                                                      | Representation        | Model Type             |\n| ------------------------------------------------------------ | --------------------- | ---------------------- |\n| [LIME](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/lime.py) | Input Features        | Model-Agnostic         |\n| [LIME with Prior](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/lime_prior.py) | Input Features        | Model-Agnostic         |\n| [GLIME](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/glime.py) | Input Features        | Model-Agnostic         |\n| [NormLIME/FastNormLIME](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/_normlime_base.py) | Input Features        | Model-Agnostic         |\n| [LRP](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/lrp.py) | Input Features        | Differentiable*        |\n| [SmoothGrad](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/smooth_grad.py) | Input Features        | Differentiable         |\n| [IntGrad](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/integrated_gradients.py) | Input Features        | Differentiable         |\n| [GradSHAP](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/gradient_shap.py) | Input Features        | Differentiable         |\n| [Occlusion](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/occlusion.py) | Input Features        | Model-Agnostic         |\n| [GradCAM/CAM](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/gradient_cam.py) | Intermediate Features | Specific: CNNs         |\n| [ScoreCAM](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/score_cam.py) | Intermediate Features | Specific: CNNs         |\n| [Rollout](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/rollout.py) | Intermediate Features | Specific: Transformers |\n| [TAM](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/transition_attention_maps.py) | Input Features        | Specific: Transformers |\n| [Generic Attention](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/generic_attention.py) | Input Features        | Specific: Transformers |\n| [Bidirectional](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/bidirectional_transformer.py) | Input Features        | Specific: Transformers |\n| [ForgettingEvents](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/forgetting_events.py) | Dataset-Level         | Differentiable         |\n| [TIDY (Training Data Analyzer)](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/TIDY.ipynb) | Dataset-Level         | Differentiable         |\n| [BHDF](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/training_dynamics.py) | Dataset-Level**       | Differentiable         |\n| [Consensus](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/consensus.py) | Features              | Cross-Model            |\n\n\\* LRP requires that the model is of specific implementations for relevance back-propagation.\n\n\\*\\* Dataset-Level Interpreters require a training process.\n\n## Implemented Trustworthiness Evaluation Algorithms\n\n- [x] [Perturbation Tests](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/evaluate_interpreter/perturbation.py)\n- [x] [Deletion \u0026 Insertion](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/evaluate_interpreter/deletion_insertion.py)\n- [x] [Infidelity](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/evaluate_interpreter/infidelity.py)\n\n## Interpretability Evaluation Algorithms\n\n- [x] [Localization Ability](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/evaluate_interpreter/localization.py)\n\n## Planning Alorithms\n\n* Intermediate Features Interpretation Algorithm\n  - [x] More Transformers Specific Interpreters\n\n* Dataset-Level Interpretation Algorithms\n  - [ ] Influence Function\n\n* Evaluations\n  - [ ] Local Fidelity\n  - [ ] Monotonicity\n  - [x] Infidelity\n  - [ ] Sensitivity\n\n# Presentations\n\n**Linux Foundation Project AI \u0026 Data** -- Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond. [Video Link](https://wiki.lfaidata.foundation/download/attachments/7733341/GMT20220324-130226_Recording_3840x2160.mp4?version=1\u0026modificationDate=1649079184753\u0026api=v2) (00:20:30 -- 00:45:00).\n\n**Baidu Create 2021 (in Chinese)**: [Video Link](https://live.baidu.com/m/media/pclive/pchome/live.html?room_id=5073321791\u0026source=h5pre) (01:18:40 -- 01:36:30).\n\n**ICML 2021 Expo** -- Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond. [Video Link](https://icml.cc/ExpoConferences/2021/workshop/11429#wse-detail-11435).\n\n# References of Algorithms\n\n* `SGDNoise`: [On the Noisy Gradient Descent that Generalizes as SGD, Wu et al 2019](https://arxiv.org/abs/1906.07405)\n* `IntegratedGraients`: [Axiomatic Attribution for Deep Networks, Mukund Sundararajan et al. 2017](https://arxiv.org/abs/1703.01365)\n* `CAM`, `GradCAM`: [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, Ramprasaath R. Selvaraju et al. 2017](https://arxiv.org/abs/1610.02391.pdf)\n* `SmoothGrad`: [SmoothGrad: removing noise by adding noise, Daniel Smilkov et al. 2017](https://arxiv.org/abs/1706.03825)\n* `GradientShap`: [A Unified Approach to Interpreting Model Predictions, Scott M. Lundberg et al. 2017](http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions)\n* `Occlusion`: [Visualizing and Understanding Convolutional Networks, Matthew D Zeiler and Rob Fergus 2013](https://arxiv.org/abs/1311.2901)\n* `Lime`: [\"Why Should I Trust You?\": Explaining the Predictions of Any Classifier, Marco Tulio Ribeiro et al. 2016](https://arxiv.org/abs/1602.04938)\n* `NormLime`: [NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks, Isaac Ahern et al. 2019](https://arxiv.org/abs/1909.04200)\n* `ScoreCAM`: [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks, Haofan Wang et al. 2020](https://arxiv.org/abs/1910.01279)\n* `ForgettingEvents`: [An Empirical Study of Example Forgetting during Deep Neural Network Learning, Mariya Toneva et al. 2019](http://arxiv.org/abs/1812.05159)\n* `LRP`: [On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation, Bach et al. 2015](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140)\n* `Rollout`: [Quantifying Attention Flow in Transformers, Abnar et al. 2020](https://arxiv.org/abs/2005.00928)\n* `TAM`: [Explaining Information Flow Inside Vision Transformers Using Markov Chain. Yuan et al. 2021](https://openreview.net/forum?id=TT-cf6QSDaQ)\n* `Consensus`: [Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study. Li et al 2021](https://arxiv.org/abs/2109.00707)\n* `Perturbation`: [Evaluating the visualization of what a deep neural network has learned.](https://arxiv.org/abs/1509.06321)\n* `Deletion\u0026Insertion`: [RISE: Randomized Input Sampling for Explanation of Black-box Models.](https://arxiv.org/abs/1806.07421)\n* `PointGame`: [Top-down Neural Attention by Excitation Backprop.](https://arxiv.org/abs/1608.00507)\n* `Generic Attention`: [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://arxiv.org/abs/2103.15679)\n\n# Copyright and License\n\nInterpretDL is provided under the [Apache-2.0 license](https://github.com/PaddlePaddle/InterpretDL/blob/master/LICENSE).\n\n# Recent News\n\n- (2022.11) Two papers got accepted by AAAI'23 and Artificial Intelligence respectively. See implementations at [G-LIME](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/glime.py) and [TrainingDynamics](https://github.com/PaddlePaddle/InterpretDL/blob/master/interpretdl/interpreter/training_dynamics.py).\n\n\u003e Xuhong Li, Haoyi Xiong, Xingjian Li, Xiao Zhang, Ji Liu, Haiyan Jiang, Zeyu Chen, Dejing Dou. “G-LIME: Statistical Learning for Local Interpretations of Deep Neural Networks using Global Priors.” Artificial Intelligence, 2023. [pdf link](https://github.com/PaddlePaddle/InterpretDL/files/10110787/glime-aij-paper.pdf). \n\n\u003e Qingrui Jia, Xuhong Li, Lei Yu, Penghao Zhao, Jiang Bian, Shupeng Li, Haoyi Xiong, Dejing Dou. “Learning from Training Dynamics: Identifying Mislabeled Data Beyond Manually Designed Features”. AAAI 2023. [pdf link](https://arxiv.org/abs/2212.09321).\n\n- (2022.08) The paper with this repository is accepted by Journal of Machine Learning Research (JMLR). If this repo is helpful for your work, please consider citing this paper:\n\n  \u003e Xuhong Li, Haoyi Xiong, Xingjian Li, Xuanyu Wu, Zeyu Chen, and Dejing Dou. “InterpretDL: Explaining Deep Models in PaddlePaddle.” Journal of Machine Learning Research, 2022. https://jmlr.org/papers/v23/21-0738.html. \n\n  ```\n  @article{JMLR:v23:21-0738,\n    author  = {Xuhong Li and Haoyi Xiong and Xingjian Li and Xuanyu Wu and Zeyu Chen and Dejing Dou},\n    title   = {InterpretDL: Explaining Deep Models in PaddlePaddle},\n    journal = {Journal of Machine Learning Research},\n    year    = {2022},\n    volume  = {23},\n    number  = {197},\n    pages   = {1--6},\n    url     = {http://jmlr.org/papers/v23/21-0738.html}\n    }\n  ```\n\n- (2022.08) Two research works got accepted by ECML and Machine Learning Journal. Cross-model consensus explanations are exploited to create a pseudo semantic segmentation dataset named PSSL, which contains 1.2M pseudo masks corresponding to the whole ImageNet training set. The dataset is publicly available. \n\n  Refer to [PaddleSeg:PSSL](https://github.com/PaddlePaddle/PaddleSeg/tree/develop/configs/pssl) for downloading the dataset and the pretrained models. \n\n  Refer to the papers for the details about the Consensus explanation and how it helps for the semantic segmentation task:\n\n  \u003e Xuhong Li, Haoyi Xiong, Siyu Huang, Shilei Ji, Dejing Dou. Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study. ECML'22, Machine Learning Journal Track. https://arxiv.org/abs/2109.00707.\n\n  \u003e Xuhong Li, Haoyi Xiong, Yi Liu, Dingfu Zhou, Zeyu Chen, Yaqing Wang, and Dejing Dou. \"Distilling ensemble of explanations for weakly-supervised pre-training of image segmentation models.\" Machine Learning (2022): 1-17. https://arxiv.org/abs/2207.03335.\n\n  ![](https://user-images.githubusercontent.com/13829174/184098740-a84cfa80-3cc0-4b73-ad57-e51bb2ce96d1.png)\n\n- (2022/04/27) A getting-started tutorial is provided. Check it from [GitHub](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/Getting_Started.ipynb) or [NBViewer](https://nbviewer.org/github/PaddlePaddle/InterpretDL/blob/master/tutorials/Getting_Started.ipynb). Usage examples have been provided for each algorithm (both Interpreter and Evaluator). We are currently preparing tutorials for easy usages of InterpretDL. Both tutorials and examples can be assessed under the [tutorial](https://github.com/PaddlePaddle/InterpretDL/tree/master/tutorials) folder.\n\n- (2022/01/06) Implemented the Cross-Model Consensus Explanation method. In brief, this method averages the explanation results from several models. Instead of interpreting individual models, this method is able to identify the discriminative features in the input data with accurate localization. See the [paper](https://arxiv.org/abs/2109.00707) for details.\n\n  * `Consensus`: Xuhong Li, Haoyi Xiong, Siyu Huang, Shilei Ji, Dejing Dou. Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study. arXiv:2109.00707.\n\n  We show a demo with six models (the last column shows the consensus explanation), while more models (around 15) could give a much better result. See the [example](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/example_consensus_cv.ipynb) for more details.\n\n  ![Consensus Result](https://user-images.githubusercontent.com/13829174/165700043-1c680494-8573-4b4a-a2d6-74ea3d14f214.png)\n\n\n- (2021/10/20) Implemented the Transition Attention Maps (TAM) explanation method for PaddlePaddle [Vision Transformers](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/ppcls/arch/backbone/model_zoo/vision_transformer.py). As always, several lines call this interpreter. See details from the [example](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/example_tam_cv_ViT.ipynb), and the [paper](https://openreview.net/forum?id=TT-cf6QSDaQ):\n\n  * `TAM`: Tingyi Yuan, Xuhong Li, Haoyi Xiong, Hui Cao, Dejing Dou. Explaining Information Flow Inside Vision Transformers Using Markov Chain. In *Neurips 2021 XAI4Debugging Workshop*. \n\n  ```python\n  import paddle\n  import interpretdl as it\n  \n  # load vit model and weights\n  # !wget -c https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch16_224_pretrained.pdparams -P assets/\n  from assets.vision_transformer import ViT_base_patch16_224\n  paddle_model = ViT_base_patch16_224()\n  MODEL_PATH = 'assets/ViT_base_patch16_224_pretrained.pdparams'\n  paddle_model.set_dict(paddle.load(MODEL_PATH))\n  \n  # Call the interpreter.\n  tam = it.TAMInterpreter(paddle_model, use_cuda=True)\n  img_path = 'samples/el1.png'\n  heatmap = tam.interpret(\n          img_path,\n          start_layer=4,\n          label=None,  # elephant\n          visual=True,\n          save_path=None)\n  heatmap = tam.interpret(\n          img_path,\n          start_layer=4,\n          label=340,  # zebra\n          visual=True,\n          save_path=None)\n  ```\n\n  |                            image                             |                           elephant                           |                            zebra                             |\n  | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |\n  | ![image](https://user-images.githubusercontent.com/13829174/139223230-66094dbf-cbc8-450c-acd8-0c0ec40c5fef.png) | ![elephant](https://user-images.githubusercontent.com/13829174/138049903-8106d879-3c70-437b-a580-cf8e9c17f974.png) | ![zebra](https://user-images.githubusercontent.com/13829174/138049895-6d52b97d-c4fd-40da-be88-f5c956cb9fcb.png) |\n\n\n- (2021/07/22) Implemented Rollout Explanations for PaddlePaddle [Vision Transformers](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/ppcls/arch/backbone/model_zoo/vision_transformer.py). See the [notebook](https://github.com/PaddlePaddle/InterpretDL/blob/master/tutorials/example_rollout_cv_ViT.ipynb) for the visualization.\n\n  ```python\n  import paddle\n  import interpretdl as it\n  \n  # wget -c https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_small_patch16_224_pretrained.pdparams -P assets/\n  from assets.vision_transformer import ViT_small_patch16_224\n  paddle_model = ViT_small_patch16_224()\n  MODEL_PATH = 'assets/ViT_small_patch16_224_pretrained.pdparams'\n  paddle_model.set_dict(paddle.load(MODEL_PATH))\n  \n  img_path = 'assets/catdog.png'\n  rollout = it.RolloutInterpreter(paddle_model, use_cuda=True)\n  heatmap = rollout.interpret(img_path, start_layer=0, visual=True)\n  ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaddlepaddle%2Finterpretdl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaddlepaddle%2Finterpretdl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaddlepaddle%2Finterpretdl/lists"}