{"id":22331077,"url":"https://github.com/rodrigobdz/lrp","last_synced_at":"2025-07-29T19:33:08.932Z","repository":{"id":44341316,"uuid":"432443018","full_name":"rodrigobdz/lrp","owner":"rodrigobdz","description":"Explain Neural Networks using Layer-Wise Relevance Propagation and evaluate the explanations using Pixel-Flipping and Area Under the Curve.","archived":false,"fork":false,"pushed_at":"2022-08-07T21:04:18.000Z","size":92184,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-15T07:42:22.902Z","etag":null,"topics":["activation-curve","area-under-curve","au-mse","auac","dataloader","ilsvrc2012","imagenet-dataset","layer-wise-relevance-propagation","lrp","pf","pixel-flipping","pruning-curve","pytorch","qualitative-evaluation","quantitative-evaluation","region-perturbation","xai","xai-library"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/lrp-pf-auc/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rodrigobdz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"license","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.bib","codeowners":null,"security":null,"support":null}},"created_at":"2021-11-27T11:31:56.000Z","updated_at":"2024-08-29T10:14:21.000Z","dependencies_parsed_at":"2022-08-25T14:00:43.535Z","dependency_job_id":null,"html_url":"https://github.com/rodrigobdz/lrp","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodrigobdz%2Flrp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodrigobdz%2Flrp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodrigobdz%2Flrp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rodrigobdz%2Flrp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rodrigobdz","download_url":"https://codeload.github.com/rodrigobdz/lrp/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228041240,"owners_count":17860221,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activation-curve","area-under-curve","au-mse","auac","dataloader","ilsvrc2012","imagenet-dataset","layer-wise-relevance-propagation","lrp","pf","pixel-flipping","pruning-curve","pytorch","qualitative-evaluation","quantitative-evaluation","region-perturbation","xai","xai-library"],"created_at":"2024-12-04T04:09:35.898Z","updated_at":"2024-12-04T04:09:36.530Z","avatar_url":"https://github.com/rodrigobdz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Layer-Wise Relevance Propagation\n\n[![DOI](https://zenodo.org/badge/432443018.svg)](https://zenodo.org/badge/latestdoi/432443018)\n[![PyPI Version](https://img.shields.io/pypi/v/lrp-pf-auc)](https://pypi.org/project/lrp-pf-auc/)\n[![License](https://img.shields.io/badge/license-MIT-blue)](license)\n\nPyTorch implementation of Layer-wise Relevance Propagation (LRP) algorithm together with quantitative evaluation metrics to compare heatmap explanations objectively part of master's thesis [_In-Depth Hyperparameter Selection For Layer-Wise Relevance Propagation_](https://zenodo.org/record/6972407#.YvAikS8Rr0o) at TU Berlin.\n\nSpecial thanks to Dr. Grégoire Montavon for his insights, which shaped the development of this project.\n\n## Features\n\nExplainability:\n\n- Layer-wise Relevance Propagation (LRP)\n\nQuantitative Evaluation:\n\n- Pixel Flipping (PF), also known as Region Perturbation (RP)\n  - Perturbation modes:\n    - inpainting\n    - random\n  - Sort objectives:\n    - most relevant first (MoRF), also known as activation curve\n    - least relevant first (LRF), also known as pruning curve\n    - random\n- AUC (Area Under the Curve), also known as AUAC (Area Under the Activation Curve) or AU-MSE (Area Under the Mean Squared Error Curve) depending on the sort objective\n\n### Showcase\n\n**LRP** ([lrp-tutorial](https://git.tu-berlin.de/gmontavon/lrp-tutorial) composite) and **PF** with **inpainting** perturbation mode and sort objective **MoRF**.\n\n\u003cp align=\"center\" width=\"100%\"\u003e\n    \u003cimg width=\"65%\" src=\"https://github.com/rodrigobdz/lrp/blob/main/docs/images/castle-lrp-pf-auc-inpainting.png?raw=true\"\u003e\n\u003c/p\u003e\n\nClassification scores of castle image with **inpainting** perturbation mode\n\n\u003cp align=\"center\" width=\"100%\"\u003e\n    \u003cimg width=\"65%\" src=\"https://github.com/rodrigobdz/lrp/blob/main/docs/images/castle-pf-auc-inpainting.png?raw=true\"\u003e\n\u003c/p\u003e\n\nNumber of simultaneous flips per perturbation step during Pixel-Flipping.\n\n\u003cp align=\"center\" width=\"100%\"\u003e\n    \u003cimg width=\"65%\" src=\"https://github.com/rodrigobdz/lrp/blob/main/docs/images/castle-pf-perturbation-steps.png?raw=true\"\u003e\n\u003c/p\u003e\n\n---\n\n\u003cdetails\u003e\n\u003csummary\u003ePF perturbation mode random\u003c/summary\u003e\n\u003cp\u003e\n\n**LRP** ([lrp-tutorial](https://git.tu-berlin.de/gmontavon/lrp-tutorial) composite) and **PF** with **random** perturbation mode and sort objective **MoRF**.\n\n\u003cp align=\"center\" width=\"100%\"\u003e\n    \u003cimg width=\"65%\" src=\"https://github.com/rodrigobdz/lrp/blob/main/docs/images/castle-lrp-pf-auc-random.png?raw=true\"\u003e\n\u003c/p\u003e\n\nClassification scores of castle image with random perturbation mode\n\n\u003cp align=\"center\" width=\"100%\"\u003e\n    \u003cimg width=\"65%\" src=\"https://github.com/rodrigobdz/lrp/blob/main/docs/images/castle-pf-auc-random.png?raw=true\"\u003e\n\u003c/p\u003e\n\n\u003c/p\u003e\n\u003c/details\u003e\n\n## Requirements\n\n- `python3` \u003e= 3.9\n\n## Installation\n\n```sh\npython3 -m pip install lrp-pf-auc\n```\n\nThe PyPI distribution `lrp-pf-auc` provides the following two packages: `lrp` and `pf`.\n\nThe name `lrp-pf-auc` stands for Layer-wise Relevance Propagation (LRP), Pixel Flipping (PF), and Area Under the Curve (AUC) respectively.\n\n## Usage\n\nRefer to [demo.ipynb](https://github.com/rodrigobdz/lrp/blob/main/demo.ipynb) for an example of Layer-wise Relevance Propagation (LRP), Pixel-Flipping (PF) and Area under the Curve (AUC).\n\nFeel free to check out the Jupyter notebooks under [experiments/notebooks](https://github.com/rodrigobdz/lrp/tree/main/experiments/notebooks) for a chronological overview of the project.\n\n## Related Projects\n\n- Sequential LRP implementation: [gmontavon/lrp-tutorial](https://git.tu-berlin.de/gmontavon/lrp-tutorial)\n  \u003e Tutorial on how to implement LRP\n- Updated version of `gmontavon/lrp-tutorial`: [rodrigobdz/lrp-tutorial](https://git.tu-berlin.de/rodrigobdz/lrp-tutorial)\n- Forward-hook LRP implementation: [chr5tphr/zennit](https://github.com/chr5tphr/zennit)\n  \u003e Implementation of LRP-based methods in PyTorch\n- [`innvestigate`](https://github.com/albermax/innvestigate)-based LRP implementation: [moboehle/Pytorch-LRP](https://github.com/moboehle/Pytorch-LRP)\n- Caffe-based LRP implementation: [`lrp_toolbox`](https://github.com/sebastian-lapuschkin/lrp_toolbox)\n- Pixel-Flipping and Region Perturbation implementation: [understandable-machine-intelligence-lab/Quantus](https://github.com/understandable-machine-intelligence-lab/Quantus)\n\n## Citation\n\nCite as:\n\n- Plaintext:\n\n  \u003e Rodrigo Bermúdez Schettino. (2022). rodrigobdz/lrp: v0.1.6 (v0.1.6). Zenodo. https://doi.org/10.5281/zenodo.6821295\n\n- BibTeX:\n\n  ```text\n  @software{rodrigo_bermudez_schettino_2022_6821295,\n    author       = {Rodrigo Bermúdez Schettino},\n    title        = {rodrigobdz/lrp: v0.1.6},\n    month        = jul,\n    year         = 2022,\n    publisher    = {Zenodo},\n    version      = {v0.1.6},\n    doi          = {10.5281/zenodo.6821295},\n    url          = {https://doi.org/10.5281/zenodo.6821295}\n  }\n  ```\n\n## Credits\n\n- The structure of this readme is based on [minimal-readme](https://github.com/rodrigobdz/minimal-readme)\n\n- Scripts follow [Personal Shell Style Guide](https://github.com/rodrigobdz/styleguide-sh)\n\n- The `lrp` package uses [two customized files](https://github.com/rodrigobdz/lrp/tree/main/lrp/zennit) originally from [chr5tphr/zennit](https://github.com/chr5tphr/zennit).\n\n- The syntax for defining custom composites is inspired by [this discussion](https://github.com/chr5tphr/zennit/issues/76) on `zennit`'s repo.\n\nThis implementation is based on insights from:\n\n- LRP overview paper\n\n  \u003e G. Montavon, A. Binder, S. Lapuschkin, W. Samek, K.-R. Müller\n  \u003e [Layer-wise Relevance Propagation: An Overview](https://doi.org/10.1007/978-3-030-28954-6_10)\n  \u003e in Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer LNCS, vol. 11700, 2019\n\n- Original LRP paper\n\n  \u003e S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, W. Samek\n  \u003e [On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation](https://doi.org/10.1371/journal.pone.0130140)\n  \u003e PloS ONE 10 (7), e0130140, 2015\n\n- [ECML/PKDD 2020 Tutorial: Explainable AI for Deep Networks: Basics and Extensions (Part 3)](http://heatmapping.org/slides/2020_ECML_3.pdf)\n\n## License\n\n[MIT](LICENSE) © [rodrigobdz](https://github.com/rodrigobdz/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frodrigobdz%2Flrp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frodrigobdz%2Flrp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frodrigobdz%2Flrp/lists"}