Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/eclique/RISE
Detect model's attention
https://github.com/eclique/RISE
classification explainable-ai saliency saliency-map xai
Last synced: about 1 month ago
JSON representation
Detect model's attention
- Host: GitHub
- URL: https://github.com/eclique/RISE
- Owner: eclique
- License: mit
- Created: 2018-09-14T01:16:33.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2020-07-02T07:16:09.000Z (almost 4 years ago)
- Last Synced: 2024-02-07T07:09:28.994Z (5 months ago)
- Topics: classification, explainable-ai, saliency, saliency-map, xai
- Language: Jupyter Notebook
- Size: 7.47 MB
- Stars: 138
- Watchers: 5
- Forks: 47
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome-stars - eclique/RISE - Detect model's attention (Jupyter Notebook)
- awesome-machine-learning-interpretability - RISE - people.bu.edu/vpetsiuk/), [Abir Das](http://cs-people.bu.edu/dasabir/), [Kate Saenko](http://ai.bu.edu/ksaenko.html) (BMVC, 2018) [and] [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://arxiv.org/abs/1806.07421).” | (Technical Resources / Open Source/Access Responsible AI Software Packages)
README
# RISE
This repository contains source code necessary to reproduce some of the main results in the paper:
[Vitali Petsiuk](http://cs-people.bu.edu/vpetsiuk/), [Abir Das](http://cs-people.bu.edu/dasabir/), [Kate Saenko](http://ai.bu.edu/ksaenko.html) (BMVC, 2018)
[RISE: Randomized Input Sampling for Explanation of Black-box Models](https://arxiv.org/abs/1806.07421)
**If you use this software in an academic article, please consider citing:**
@inproceedings{Petsiuk2018rise,
title = {RISE: Randomized Input Sampling for Explanation of Black-box Models},
author = {Vitali Petsiuk and Abir Das and Kate Saenko},
booktitle = {Proceedings of the British Machine Vision Conference (BMVC)},
year = {2018}
}
For more information regarding the paper, please visit http://cs-people.bu.edu/vpetsiuk/rise/.
## Method overview
To generate a saliency map for model's prediction, RISE queries black-box model on multiple randomly masked versions of input.
After all the queries are done we average all the masks with respect to their scores to produce the final saliency map. The idea behind this is that whenever a mask preserves important parts of the image it gets higher score, and consequently has a higher weight in the sum.
## Repository contents
* The whole idea is implemented in [Easy_start](Easy_start.ipynb) notebook, it's done in Keras but is really easy to modify for any framework, since the method itself is model-agnostic.
* [Saliency](Saliency.ipynb) notebook demonstrates the usage of RISE class optimized for PyTorch.
* [Evaluation](Evaluation.ipynb) notebook displays another contribution of the paper: *Causal metrics*.
## Examples
