https://github.com/jhoelli/awesome-time-series-explainability
A list of (post-hoc) XAI for time series
https://github.com/jhoelli/awesome-time-series-explainability
List: awesome-time-series-explainability
awesome-list awesome-lists explainability explainable-ai explainable-ml explainable-time-series-classification explainable-time-series-prediction interpretability interpretable-machine-learning time-series
Last synced: 2 days ago
JSON representation
A list of (post-hoc) XAI for time series
- Host: GitHub
- URL: https://github.com/jhoelli/awesome-time-series-explainability
- Owner: JHoelli
- License: mit
- Created: 2023-11-27T08:21:14.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-09-04T08:58:54.000Z (6 months ago)
- Last Synced: 2025-02-19T17:09:41.578Z (2 days ago)
- Topics: awesome-list, awesome-lists, explainability, explainable-ai, explainable-ml, explainable-time-series-classification, explainable-time-series-prediction, interpretability, interpretable-machine-learning, time-series
- Homepage:
- Size: 424 KB
- Stars: 112
- Watchers: 5
- Forks: 15
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING
- License: LICENSE
Awesome Lists containing this project
- ultimate-awesome - awesome-time-series-explainability - A list of (post-hoc) XAI for time series. (Other Lists / Julia Lists)
README
# Awesome-Time-Series-Explainability
A list of XAI for time series. This list focuses (currently) on Post-Hoc Explainability for time series data, including paper and github links. The list is expanded and updated gradually. Feel Free to update missing or new paper.
## Outline
- [Surveys](https://github.com/JHoelli/Awesome-Time-Series-Explainability#Surveys)
- [Libraries ](https://github.com/JHoelli/Awesome-Time-Series-Explainability#Libraries)
- [Classification ](https://github.com/JHoelli/Awesome-Time-Series-Explainability#Classification)
- [Regression / Forecasting](https://github.com/JHoelli/Awesome-Time-Series-Explainability#Regression-/-Forecasting)
- [Classification and Regression / Forcasting](https://github.com/JHoelli/Awesome-Time-Series-Explainability#Classification-and-Regression-/-Forcasting)
- [Benchmarking and Evaluation](https://github.com/JHoelli/Awesome-Time-Series-Explainability#Benchmarking-and-Evaluation)
- [Ante-Hoc Explanations](https://github.com/JHoelli/Awesome-Time-Series-Explainability#Ante-Hoc-Explanation)
## Surveys
- [**Explainable artificial intelligence (XAI) in finance: a systematic literature review**](https://link.springer.com/article/10.1007/s10462-024-10854-8) , (2024) by *Černevičienė, J., & Kabašinskas, A*
- [**A Survey of Explainable Artificial Intelligence (XAI) in Financial Time Series Forecasting**](https://arxiv.org/pdf/2407.15909) , (2024) by *Arsenault, P. D., Wang, S., & Patenande, J. M.*
- [**Post Hoc Explainability for Time Series Classification: Toward a signal processing perspective**](https://ieeexplore.ieee.org/document/9810094) , (2022) by *R. Mochaourab, A. Venkitaraman, I. Samsten, P. Papapetrou and C. R. Rojas*
- [**Explainable AI for time series classification: a review, taxonomy and research directions**](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9895252) , (2022) by *A Theissler, F Spinnato, U Schlegel, R Guidotti*
- [**Explainable artificial intelligence (xai) on timeseries data: A survey**](https://arxiv.org/abs/2104.00950) , (2021) by *Rojat, T., Puget, R., Filliat, D., Del Ser, J., Gelin, R., & Díaz-Rodríguez, N.*
- [**XAI Methods for Neural Time Series Classification: A Brief Review**](https://arxiv.org/abs/2108.08009) , (2021) by *Simic, I., Sabol, V., Veas, E.*
## Libraries
- [**TSInterpret: A Python Package for the Interpretability of Time Series Classification**](https://joss.theoj.org/papers/10.21105/joss.05220.pdf) (2023) by *Höllig, J., Kulbach, C., & Thoma, S.* [https://github.com/fzi-forschungszentrum-informatik/TSInterpret](https://github.com/fzi-forschungszentrum-informatik/TSInterpret), 
- [**Time Interpret: a Unified Model Interpretability Library for Time Series**](https://arxiv.org/abs/2306.02968) (2023) by *Enguehard, J.* [https://github.com/josephenguehard/time_interpret](https://github.com/josephenguehard/time_interpret), 
## Classification
### Feature Attribution
- [**Time is Not Enough: Time-Frequency based Explanation for
Time-Series Black-Box Models**](https://arxiv.org/pdf/2408.03636), (2024), by *Chung, H., Jo, S., Kwon, Y., & Choi, E. * [https://github.com//gustmd0121/time_is_not_enough](https://github.com/gustmd0121/time_is_not_enough), 
- [**Translating Image XAI to Multivariate Time Series**](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10439172), (2024), by *Tronchin, L., Cordelli, E., Celsi, L. R., Maccagnola, D., Natale, M., Soda, P., & Sicilia, R.* [https://github.com//ltronchin/translating-xai-mts](https://github.com/ltronchin/translating-xai-mts), 
- [**Explaining time series classifiers through meaningful perturbation and optimisation**](https://doi.org/10.1016/j.ins.2023.119334), (2023), by *H Meng, C Wagner, I Triguero* [https://github.com/menghan1994/ETSC_through_Meainingful_Perturbation_and_Optimisation](https://github.com/menghan1994/ETSC_through_Meainingful_Perturbation_and_Optimisation), 
- [**Explainable AI for Time Series via Virtual Inspection Layers**](https://arxiv.org/pdf/2303.06365) , (2023) by *Vielhaben, J., Lapuschkin, S., Montavon, G., & Samek, W.*
- [**LIMESegment: Meaningful, Realistic Time Series Explanations**](https://proceedings.mlr.press/v151/sivill22a.html) , (2022) by *Sivill, T., & Flach, P.*, [https://github.com/TortySivill/LIMESegment](https://github.com/TortySivill/LIMESegment), 
- [**Class-Specific Explainability for Deep Time Series Classifiers**](https://arxiv.org/pdf/2210.05411.pdf) , (2022) by *Doddaiah, R., Parvatharaju, P., Rundensteiner, E., & Hartvigsen, T.*, [https://github.com/rameshdoddaiah/DEMUX](https://github.com/rameshdoddaiah/DEMUX), 
- [**TSInsight: A local-global attribution framework for interpretability in time series data**](https://www.mdpi.com/1424-8220/21/21/7373)(2021) by *Siddiqui, S. A., Mercier, D., Dengel, A., & Ahmed, S.*
- [**Benchmarking Deep Learning Interpretability in Time Series Predictions**](https://arxiv.org/abs/2010.13924) (2020) by *Ismail, A. A., Gunady, M., Corrada Bravo, H., & Feizi, S.* [https://github.com/ayaabdelsalam91/TS-Interpretability-Benchmark](https://github.com/ayaabdelsalam91/TS-Interpretability-Benchmark), 
- [**What went wrong and when? Instance-wise feature importance for time-series black-box models**](https://papers.nips.cc/paper_files/paper/2020/file/08fa43588c2571ade19bc0fa5936e028-Paper.pdf) (2020) by *Tonekaboni, S., Joshi, S., Campbell, K., Duvenaud, D. K., & Goldenberg, A.* [https://github.com/sanatonek/time_series_explainability](https://github.com/sanatonek/time_series_explainability), 
- [**Agnostic Local Explanation for Time Series Classification**](https://ieeexplore.ieee.org/document/8995349) (2019) by *Guillemé, M., Masson, V., Rozé, L., & Termier, A. *
- [**timeXplain -- A Framework for Explaining the Predictions of Time Series Classifiers**](https://arxiv.org/abs/2007.07606) (2019) by *Mujkanovic, F., Doskoč, V., Schirneck, M., Schäfer, P., & Friedrich, T.* [https://github.com/LoadingByte/timeXplain](https://github.com/LoadingByte/timeXplain), 
- [**Tsxplain: Demystification of dnn decisions for time-series using natural language and statistical features**](https://link.springer.com/chapter/10.1007/978-3-030-30493-5_43) (2019) by *Munir, M., Siddiqui, S. A., Küsters, F., Mercier, D., Dengel, A., & Ahmed, S.*
### Counterfactuals
- [**Sub-SpaCE: Subsequence-Based Sparse Counterfactual Explanations for Time Series Classification Problems**](https://link.springer.com/chapter/10.1007/978-3-031-63800-8_1) (2024) by *Refoyo, M., & Luengo, D.* [https://github.com/MarioRefoyo/Sub-SpaCE](https://github.com/MarioRefoyo/Sub-SpaCE), 
- [**CELS: Counterfactual Explanations for Time Series Data via Learned Saliency Maps**](https://ieeexplore.ieee.org/abstract/document/10386229) (2023) by *Li, P., Bahri, O., Boubrahimi, S. F., & Hamdi, S. M.* [https://github.com/Luckilyeee/CELS](https://github.com/Luckilyeee/CELS), 
- [**Glacier: guided locally constrained counterfactual
explanations for time series classification**](https://link.springer.com/content/pdf/10.1007/s10994-023-06502-x.pdf) (2024) by *Wang, Z., Samsten, I., Miliou, I., Mochaourab, R., & Papapetrou, P.* [https://github.com/zhendong3wang/learning-time-series-counterfactuals](https://github.com/zhendong3wang/learning-time-series-counterfactuals), 
- [**Attention-Based Counterfactual Explanation for Multivariate Time Series**](https://link.springer.com/chapter/10.1007/978-3-031-39831-5_26) (2023) by *Li, P., Boubrahimi, S. F., & Hamdi, S. M.* [https://sites.google.com/view/attention-based-cf](https://sites.google.com/view/attention-based-cf)
- [**Shapelet-Based Counterfactual Explanations for Multivariate Time Series**](https://arxiv.org/abs/2208.10462) (2022) by *Bahri, O., Boubrahimi, S. F., & Hamdi, S. M.* [https://github.com/omarbahri/SETS](https://github.com/omarbahri/SETS), 
- [**TSEvo: Evolutionary counterfactual explanations for time series classification**](https://ieeexplore.ieee.org/abstract/document/10069160) (2022) by *Höllig, J., Kulbach, C., & Thoma, S.* [https://github.com/JHoelli/TSEvo](https://github.com/JHoelli/TSEvo), 
- [**Counterfactual explanations for multivariate time series**](https://ieeexplore.ieee.org/document/9462056) (2021) by *Ates, E., Aksar, B., Leung, V. J., & Coskun, A. K.* [https://github.com/peaclab/CoMTE](https://github.com/peaclab/CoMTE), 
- [**Motif-Guided Time Series Counterfactual Explanations**](https://link.springer.com/chapter/10.1007/978-3-031-37731-0_16) (2022) by *Li, P., Boubrahimi, S. F., & Hamdi, S. M.* [https://github.com/Luckilyeee/Motif-guided-counterfactual-explanation](https://github.com/Luckilyeee/Motif-guided-counterfactual-explanation), 
- [**Instance-based Counterfactual Explanations for Time Series Classification**](https://arxiv.org/abs/2009.13211) (2020) by *Delaney, E., Greene, D., & Keane, M. T.* [https://github.com/e-delaney/Instance-Based_CFE_TSC](https://github.com/e-delaney/Instance-Based_CFE_TSC), 
### Frameworks (Multiple Explanation Types)
- [**ExTea: An Evolutionary Algorithm-Based Approach for Enhancing Explainability in Time-Series Models**](https://link.springer.com/chapter/10.1007/978-3-031-70381-2_27) (2024) by *Huang, Y., Zhou, Y., Zhao, H., Fang, L., Riedel, T., & Beigl, M* [https://github.com/HuangYiran/extea](https://github.com/HuangYiran/extea), 
- [**Understanding Any Time Series Classifier with a Subsequence-based Explainer**](https://dl.acm.org/doi/pdf/10.1145/3624480) , (2023) by *Spinnato, F., Guidotti, R., Monreale, A., Nanni, M., Pedreschi, D., & Giannotti, F.* [https://github.com/fspinna/lasts](https://github.com/fspinna/lasts), 
- [**TimeSHAP: Explaining Recurrent Models through Sequence Perturbations**](https://arxiv.org/abs/2012.00073), (2020) by *Bento, J., Saleiro, P., Cruz, A. F., Figueiredo, M. A., & Bizarro, P.*, [https://github.com/feedzai/timeshap](https://github.com/feedzai/timeshap), 
## Forecasting
- [**ShapTime: A General XAI Approach for Explainable Time Series Forecasting**](https://link.springer.com/chapter/10.1007/978-3-031-47721-8_45) (2024) by *Zhang, Y., Sun, Q., Qi, D., Liu, J., Ma, R., & Petrosian, O.* [https://github.com/Zhangyuyi-0825/ShapTime](https://github.com/Zhangyuyi-0825/ShapTime), 
- [**TsSHAP: Robust model agnostic feature-based explainability for univariate time series forecasting**](https://arxiv.org/pdf/2303.12316.pdf) (2023) by *Raykar, V. C., Jati, A., Mukherjee, S., Aggarwal, N., Sarpatwar, K., Ganapavarapu, G., & Vaculin, R.*
- [**Counterfactual Explanations for Time Series Forecasting**](https://arxiv.org/abs/2310.08137) (2023) by *Wang, Z., Miliou, I., Samsten, I., & Papapetrou, P.* [https://github.com/zhendong3wang/counterfactual-explanations-for-forecasting](https://github.com/zhendong3wang/counterfactual-explanations-for-forecasting), 
- [**TEMPORAL DEPENDENCIES IN FEATURE IMPORTANCE FOR TIME SERIES PREDICTION**](https://www.cs.toronto.edu/~mvolkovs/ICLR23_WinIT.pdf) (2023) by *Leung, K. K., Rooke, C., Smith, J., Zuberi, S., & Volkovs, M.* [https://github.com/layer6ai-labs/WinIT](https://github.com/layer6ai-labs/WinIT), 
- [**TS-MULE: Local Interpretable Model-Agnostic Explanations for Time Series Forecast Models**](https://link.springer.com/chapter/10.1007/978-3-030-93736-2_1) (2021) by *Schlegel, U., Vo, D. L., Keim, D. A., & Seebacher, D.* [https://github.comdbvis-ukon/ts-mule](https://github.com/dbvis-ukon/ts-mule), 
- [**Explaining time series predictions with dynamic masks**](http://proceedings.mlr.press/v139/crabbe21a/crabbe21a.pdf) (2021) by *Crabbé, J., & Van Der Schaar, M.* [https://github.com/JonathanCrabbe/Dynamask](https://github.com/JonathanCrabbe/Dynamask), 
- [**Series saliency: Temporal interpretation for multivariate time series forecasting**](https://arxiv.org/pdf/2012.09324.pdf) (2020) by *Pan, Q., Hu, W., & Zhu, J.*
## Classification and Regression / Forcasting
- [**Tsviz: Demystification of deep learning models for time-series analysis**](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8695734) (2019) by *Siddiqui, S. A., Mercier, D., Munir, M., Dengel, A., & Ahmed, S.* [https://github.com/shoaibahmed/TSViz-Core](https://github.com/shoaibahmed/TSViz-Core), 
- [**Learning Perturbations to Explain Time Series Predictions**](https://arxiv.org/pdf/2305.18840.pdf) (2023) by *Enguehard, J.* [https://github.com/josephenguehard/time_interpret](https://github.com/josephenguehard/time_interpret), 
## Benchmarking and Evaluation
- [**XAI for Time Series Classification: Evaluating the Benefits of Model Inspection for End-Users**](https://link.springer.com/chapter/10.1007/978-3-031-63800-8_22), (2024) by *Håvardstun, B., Ferri, C., Flikka, K., & Telle, J. A*
- [**A Deep Dive into Perturbations as Evaluation Technique for Time Series XAI**](https://arxiv.org/pdf/2307.05104.pdf), (2023) by *Schlegel, U., & Keim, D. A.*, [https://github.com/visual-xai-for-time-series/time-series-xai-perturbation-analysis](https://github.com/visual-xai-for-time-series/time-series-xai-perturbation-analysis), 
- [**Evaluation of post-hoc interpretability methods in time-series classification**](https://www.nature.com/articles/s42256-023-00620-w), (2023) by *Turbé, H., Bjelogrlic, M., Lovis, C. et al.*, [https://github.com/hturbe/InterpretTime](https://github.com/hturbe/InterpretTime), 
- [**A Deep Dive into Perturbations as Evaluation Technique for Time Series XAI**](https://arxiv.org/pdf/2307.05104), (2023) by *Schlegel, U., & Keim, D. A.*
- [**Introducing the Attribution Stability Indicator: a Measure for Time Series XAI Attributions**](https://arxiv.org/abs/2310.04178), (2023) by *Schlegel, U., & Keim, D. A. *
- [**Robust Framework for Explanation Evaluation in Time Series Classification**](https://arxiv.org/abs/2306.05501), (2023) by *Nguyen, T. T., Nguyen, T. L., & Ifrim, G.* [https://github.com/mlgig/amee](https://github.com/mlgig/amee), 
- [**Evaluating Explanation Methods for Multivariate Time Series Classification**](https://arxiv.org/abs/2308.15223), (2023) by *Serramazza, D. I., Nguyen, T. T., Nguyen, T. L., & Ifrim, G. * [https://github.com/mlgig/Evaluating-Explanation-Methods-for-MTSC](https://github.com/mlgig/Evaluating-Explanation-Methods-for-MTSC), 
- [**XTSC-Bench: Quantitative Benchmarking for Explainers on Time Series Classification**](https://arxiv.org/pdf/2310.14957.pdf), (2023) by *Höllig, J., Thoma, S., & Grimm, F.* [https://github.com/JHoelli/XTSC-Bench](https://github.com/JHoelli/XTSC-Bench), 
- [**Time to Focus: A Comprehensive Benchmark Using Time Series Attribution Methods**](https://arxiv.org/abs/2202.03759), (2022) by *Mercier, D., Bhatt, J., Dengel, A., & Ahmed, S.*
- [**Exathlon: A Benchmark for Explainable Anomaly Detection over Time Series**](https://arxiv.org/pdf/2010.05073.pdf), (2021) by *Jacob, V., Song, F., Stiegler, A., Rad, B., Diao, Y., & Tatbul, N.* [https://github.com/exathlonbenchmark/exathlon](https://github.com/exathlonbenchmark/exathlon), 
- [**Towards a rigorous evaluation of XAI methods on time series**](https://ieeexplore.ieee.org/document/9022428), (2019) by *Schlegel, U., Arnout, H., El-Assady, M., Oelke, D., & Keim, D. A.*
## Ante-Hoc Explanation
While this repository mostly focuses on post-hoc explanations - i.e. introducing the explanations after the training of the predictor, this section includes some approches that include explanability into the predicors design (e.g., via architecture or training).
### Classification
- [**Fast, accurate and explainable time series classification through randomization**](https://link.springer.com/article/10.1007/s10618-023-00978-w) (2023) by *Cabello, N., Naghizade, E., Qi, J., & Kulik, L.* [https://github.com/stevcabello/r-STSF](https://github.com/stevcabello/r-STSF), 
- [**XEM: An explainable-by-design ensemble method for multivariate time series classification**](https://link.springer.com/article/10.1007/s10618-022-00823-6) (2022) by *Fauvel, K., Fromont, É., Masson, V., Faverdin, P., & Termier, A* [https://github.com/XAIseries/XEM](https://github.com/XAIseries/XEM), 
- [**Xcm: An explainable convolutional neural network for multivariate time series classification**](https://www.mdpi.com/2227-7390/9/23/3137) (2021) by *Fauvel, K., Lin, T., Masson, V., Fromont, É., & Termier, A.* [https://github.com/XAIseries/XCM](https://github.com/XAIseries/XCM), 
- [**Explaining Deep Classification of Time-Series Data with Learned Prototypes**](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8050893/) (2019) by *Gee, A. H., Garcia-Olano, D., Ghosh, J., & Paydarfar, D.* [https://github.com/alangee/ijcai19-ts-prototypes](https://github.com/alangee/ijcai19-ts-prototypes), 
- [**Medical Time Series Classification with Hierarchical Attention-based Temporal Convolutional Networks: A Case Study of Myotonic Dystrophy Diagnosis**](https://openaccess.thecvf.com/content_CVPRW_2019/papers/Explainable%20AI/Lin_Medical_Time_Series_Classification_with_Hierarchical_Attention-based_Temporal_Convolutional_Networks_CVPRW_2019_paper.pdf) (2019) by *Lin, L., Xu, B., Wu, W., Richardson, T. W., & Bernal, E. A.*
- [**Explainable Failure Predictions with RNN Classifiers based on Time Series Data**](https://arxiv.org/abs/1901.08554) (2019) by *Giurgiu, I., & Schumann, A.*
- [**MTEX-CNN: Multivariate Time Series EXplanations for Predictions with Convolutional Neural Networks**](https://ieeexplore.ieee.org/document/8970899) (2019) by *Assaf, R., Giurgiu, I., Bagehorn, F., & Schumann, A.*
- [**Retain: An interpretable predictive model for healthcare using reverse time attention mechanism**](https://proceedings.neurips.cc/paper/2016/hash/231141b34c82aa95e48810a9d1b33a79-Abstract.html) (2016) by *Choi, E., Bahadori, M. T., Sun, J., Kulas, J., Schuetz, A., & Stewart, W.* [https://github.com/mp2893/retain](https://github.com/mp2893/retain), 
### Forecasting
- [**Temporal fusion transformers for interpretable multi-horizon time series forecasting**](https://www.sciencedirect.com/science/article/pii/S0169207021000637) (2021) by *Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T.*
- [**Interpretable Multivariate Time Series Forecasting with Temporal Attention Convolutional Neural Networks**](https://research.vu.nl/en/publications/interpretable-multivariate-time-series-forecasting-with-temporal-) (2020) by *Pantiskas, L., Verstoep, K., & Bal, H.* [https://github.com/lpphd/multivariate-attention-tcn](https://github.com/lpphd/multivariate-attention-tcn), 
- [**Exploring interpretable LSTM neural networks over multi-variable data**](http://proceedings.mlr.press/v97/guo19b.html) (2019) by *Guo, Tian, Tao Lin, and Nino Antulov-Fantulin* [https://github.com/weilai0980/IMV-LSTM](https://github.com/weilai0980/IMV-LSTM), 
- [**A memory-network based solution for multivariate time-series forecasting**](https://arxiv.org/abs/1809.02105) (2018) by *Chang, Y. Y., Sun, F. Y., Wu, Y. H., & Lin, S. D.*