https://github.com/PAIR-code/saliency
Framework-agnostic implementation for state-of-the-art saliency methods (XRAI, BlurIG, SmoothGrad, and more).
https://github.com/PAIR-code/saliency
convolutional-neural-networks deep-learning deep-neural-networks ig-saliency image-recognition machine-learning object-detection saliency saliency-map smoothgrad tensorflow
Last synced: 29 days ago
JSON representation
Framework-agnostic implementation for state-of-the-art saliency methods (XRAI, BlurIG, SmoothGrad, and more).
- Host: GitHub
- URL: https://github.com/PAIR-code/saliency
- Owner: PAIR-code
- License: apache-2.0
- Created: 2017-06-09T22:07:35.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2024-03-20T19:44:13.000Z (about 1 year ago)
- Last Synced: 2024-05-22T07:52:45.779Z (about 1 year ago)
- Topics: convolutional-neural-networks, deep-learning, deep-neural-networks, ig-saliency, image-recognition, machine-learning, object-detection, saliency, saliency-map, smoothgrad, tensorflow
- Language: Jupyter Notebook
- Homepage: https://pair-code.github.io/saliency/
- Size: 314 MB
- Stars: 934
- Watchers: 24
- Forks: 190
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- StarryDivineSky - PAIR-code/saliency
- Awesome-explainable-AI - https://github.com/PAIR-code/saliency - code/saliency?style=social) (Python Libraries(sort in alphabeta order) / Evaluation methods)
- awesome-python-machine-learning-resources - GitHub
README
# Saliency Library
## Updates
🔴 Now framework-agnostic! [(Example core notebook)](Examples_core.ipynb) 🔴
🔗 For further explanation of the methods and more examples of the resulting maps, see our [Github Pages website](https://pair-code.github.io/saliency) 🔗
If upgrading from an older version, update old imports to `import saliency.tf1 as saliency`. We provide wrappers to make the framework-agnostic version compatible with TF1 models. [(Example TF1 notebook)](Examples_tf1.ipynb)
🔴 Added Performance Information Curve (PIC) - a human
independent metric for evaluating the quality of saliency methods.
([Example notebook](https://github.com/PAIR-code/saliency/blob/master/pic_metrics.ipynb)) 🔴
## Saliency Methods
This repository contains code for the following saliency techniques:
* Guided Integrated Gradients* ([paper](https://arxiv.org/abs/2106.09788), [poster](https://github.com/PAIR-code/saliency/blob/master/docs/CVPR_Guided_IG_Poster.pdf))
* XRAI* ([paper](https://arxiv.org/abs/1906.02825), [poster](https://github.com/PAIR-code/saliency/blob/master/docs/ICCV_XRAI_Poster.pdf))
* SmoothGrad* ([paper](https://arxiv.org/abs/1706.03825))
* Vanilla Gradients
([paper](https://scholar.google.com/scholar?q=Visualizing+higher-layer+features+of+a+deep+network&btnG=&hl=en&as_sdt=0%2C22),
[paper](https://arxiv.org/abs/1312.6034))
* Guided Backpropogation ([paper](https://arxiv.org/abs/1412.6806))
* Integrated Gradients ([paper](https://arxiv.org/abs/1703.01365))
* Occlusion
* Grad-CAM ([paper](https://arxiv.org/abs/1610.02391))
* Blur IG ([paper](https://arxiv.org/abs/2004.03383))
\*Developed by PAIR.
This list is by no means comprehensive. We are accepting pull requests to add
new methods!
## Evaluation of Saliency Methods
The repository provides an implementation of Performance Information Curve (PIC) -
a human independent metric for evaluating the quality of saliency methods
([paper](https://arxiv.org/abs/1906.02825),
[poster](https://github.com/PAIR-code/saliency/blob/master/docs/ICCV_XRAI_Poster.pdf),
[code](https://github.com/PAIR-code/saliency/blob/master/saliency/metrics/pic.py),
[notebook](https://github.com/PAIR-code/saliency/blob/master/pic_metrics.ipynb)).
## Download
```
# To install the core subpackage:
pip install saliency
# To install core and tf1 subpackages:
pip install saliency[tf1]
```
or for the development version:
```
git clone https://github.com/pair-code/saliency
cd saliency
```
## Usage
The saliency library has two subpackages:
* `core` uses a generic `call_model_function` which can be used with any ML
framework.
* `tf1` accepts input/output tensors directly, and sets up the necessary
graph operations for each method.
### Core
Each saliency mask class extends from the `CoreSaliency` base class. This class
contains the following methods:
* `GetMask(x_value, call_model_function, call_model_args=None)`: Returns a mask
of
the shape of non-batched `x_value` given by the saliency technique.
* `GetSmoothedMask(x_value, call_model_function, call_model_args=None, stdev_spread=.15, nsamples=25, magnitude=True)`:
Returns a mask smoothed of the shape of non-batched `x_value` with the
SmoothGrad technique.
The visualization module contains two methods for saliency visualization:
* ```VisualizeImageGrayscale(image_3d, percentile)```: Marginalizes across the
absolute value of each channel to create a 2D single channel image, and clips
the image at the given percentile of the distribution. This method returns a
2D tensor normalized between 0 to 1.
* ```VisualizeImageDiverging(image_3d, percentile)```: Marginalizes across the
value of each channel to create a 2D single channel image, and clips the
image at the given percentile of the distribution. This method returns a
2D tensor normalized between -1 to 1 where zero remains unchanged.
If the sign of the value given by the saliency mask is not important, then use
```VisualizeImageGrayscale```, otherwise use ```VisualizeImageDiverging```. See
the SmoothGrad paper for more details on which visualization method to use.
##### call_model_function
`call_model_function` is how we pass inputs to a given model and receive the outputs
necessary to compute saliency masks. The description of this method and expected
output format is in the `CoreSaliency` description, as well as separately for each method.
##### Examples
[This example iPython notebook](http://github.com/pair-code/saliency/blob/master/Examples_core.ipynb)
showing these techniques is a good starting place.
Here is a condensed example of using IG+SmoothGrad with TensorFlow 2:
```
import saliency.core as saliency
import tensorflow as tf
...
# call_model_function construction here.
def call_model_function(x_value_batched, call_model_args, expected_keys):
tape = tf.GradientTape()
grads = np.array(tape.gradient(output_layer, images))
return {saliency.INPUT_OUTPUT_GRADIENTS: grads}
...
# Load data.
image = GetImagePNG(...)
# Compute IG+SmoothGrad.
ig_saliency = saliency.IntegratedGradients()
smoothgrad_ig = ig_saliency.GetSmoothedMask(image,
call_model_function,
call_model_args=None)
# Compute a 2D tensor for visualization.
grayscale_visualization = saliency.VisualizeImageGrayscale(
smoothgrad_ig)
```
### TF1
Each saliency mask class extends from the `TF1Saliency` base class. This class
contains the following methods:
* `__init__(graph, session, y, x)`: Constructor of the SaliencyMask. This can
modify the graph, or sometimes create a new graph. Often this will add nodes
to the graph, so this shouldn't be called continuously. `y` is the output
tensor to compute saliency masks with respect to, `x` is the input tensor
with the outer most dimension being batch size.
* `GetMask(x_value, feed_dict)`: Returns a mask of the shape of non-batched
`x_value` given by the saliency technique.
* `GetSmoothedMask(x_value, feed_dict)`: Returns a mask smoothed of the shape
of non-batched `x_value` with the SmoothGrad technique.
The visualization module contains two visualization methods:
* ```VisualizeImageGrayscale(image_3d, percentile)```: Marginalizes across the
absolute value of each channel to create a 2D single channel image, and clips
the image at the given percentile of the distribution. This method returns a
2D tensor normalized between 0 to 1.
* ```VisualizeImageDiverging(image_3d, percentile)```: Marginalizes across the
value of each channel to create a 2D single channel image, and clips the
image at the given percentile of the distribution. This method returns a
2D tensor normalized between -1 to 1 where zero remains unchanged.
If the sign of the value given by the saliency mask is not important, then use
```VisualizeImageGrayscale```, otherwise use ```VisualizeImageDiverging```. See
the SmoothGrad paper for more details on which visualization method to use.
##### Examples
[This example iPython notebook](http://github.com/pair-code/saliency/blob/master/Examples_tf1.ipynb) shows
these techniques is a good starting place.
Another example of using GuidedBackprop with SmoothGrad from TensorFlow:
```
from saliency.tf1 import GuidedBackprop
from saliency.tf1 import VisualizeImageGrayscale
import tensorflow.compat.v1 as tf
...
# Tensorflow graph construction here.
y = logits[5]
x = tf.placeholder(...)
...
# Compute guided backprop.
# NOTE: This creates another graph that gets cached, try to avoid creating many
# of these.
guided_backprop_saliency = GuidedBackprop(graph, session, y, x)
...
# Load data.
image = GetImagePNG(...)
...
smoothgrad_guided_backprop =
guided_backprop_saliency.GetMask(image, feed_dict={...})
# Compute a 2D tensor for visualization.
grayscale_visualization = visualization.VisualizeImageGrayscale(
smoothgrad_guided_backprop)
```
## Conclusion/Disclaimer
If you have any questions or suggestions for improvements to this library,
please contact the owners of the `PAIR-code/saliency` repository.
This is not an official Google product.