Abstract:Time series classification is a task which deals with temporal sequences, a prevalent data type common in domains such as human activity recognition, sports analytics and general sensing. In this area, interest in explanability has been growing as explanation is key to understand the data and the model better. Recently, a great variety of techniques (e.g., LIME, SHAP, CAM) have been proposed and adapted for time series to provide explanation in the form of saliency maps , where the importance of each data point in the time series is quantified with a numerical value. However, the saliency maps can and often disagree, so it is unclear which one to use. This paper provides a novel framework to quantitatively evaluate and rank explanation methods for time series classification . We show how to robustly evaluate the informativeness of a given explanation method (i.e., relevance for the classification task), and how to compare explanations side-by-side. The goal is to recommend the best explainer for a given time series classification dataset. We propose AMEE, a Model-Agnostic Explanation Evaluation framework, for recommending saliency-based explanations for time series classification. In this approach, data perturbation is added to the input time series guided by each explanation. Our results show that perturbing discriminative parts of the time series leads to significant changes in classification accuracy, which can be used to evaluate each explanation. To be robust to different types of perturbations and different types of classifiers, we aggregate the accuracy loss across perturbations and classifiers. This novel approach allows us to recommend the best explainer among a set of different explainers, including random and oracle explainers. We provide a quantitative and qualitative analysis for synthetic datasets, a variety of time-series datasets, as well as a real-world case study with known expert ground truth.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper primarily addresses the issue of evaluating and recommending explanation methods in time series classification. Specifically: 1. **Proposing the AMEE Framework**: - The paper introduces a framework called AMEE (Model-Agnostic Explanation Evaluation) for evaluating and ranking explanation methods in time series classification. This framework aims to recommend the best explainers, thereby helping users better understand the data and models. 2. **Standardized Evaluation Measures**: - It proposes a standardized evaluation metric (explanation capability) that can be compared across different explanation methods, classifier judges, and datasets. 3. **Experimental Validation**: - Experiments were conducted on synthetic and real datasets, and the evaluation method's effectiveness was validated using annotated real datasets. All data, code, and detailed results are publicly available. 4. **Problem Solved**: - In time series classification tasks, the saliency maps obtained from various explanation methods are often inconsistent, necessitating a method to evaluate and compare the effectiveness of these explanation methods. The paper addresses this issue through the AMEE framework, providing an objective evaluation standard. Through these contributions, the paper addresses the shortcomings in the current evaluation of explanation methods in time series classification, offering a more reliable and general evaluation framework.

Robust explainer recommendation for time series classification

Robust Explainer Recommendation for Time Series Classification

Ranking by Aggregating Referees: Evaluating the Informativeness of Explanation Methods for Time Series Classification

Improving the Evaluation and Actionability of Explanation Methods for Multivariate Time Series Classification

XCM: an Explainable Convolutional Neural Network for Multivariate Time Series Classification.

Explaining deep multi-class time series classifiers

Time is Not Enough: Time-Frequency based Explanation for Time-Series Black-Box Models

SEGAL time series classification - Stable explanations using a generative model and an adaptive weighting method for LIME

Visual Explanations with Attributions and Counterfactuals on Time Series Classification

TimeX++: Learning Time-Series Explanations with Information Bottleneck

Robust Explainable Recommendation

PUPAE: Intuitive and Actionable Explanations for Time Series Anomalies

Explainable AI for Time Series Classification: A Review, Taxonomy and Research Directions

Explanation Space: A New Perspective into Time Series Interpretability

SSET: Swapping-Sliding Explanation for Time Series Classifiers in Affect Detection

TS-MULE: Local Interpretable Model-Agnostic Explanations for Time Series Forecast Models

Towards Faithful Explanations for Text Classification with Robustness Improvement and Explanation Guided Training

Generating Explanations for Explainable Recommendations Using Filter-Enhanced Time-Series Information

Stability of Explainable Recommendation

CAFO: Feature-Centric Explanation on Time Series Classification

XForecast: Evaluating Natural Language Explanations for Time Series Forecasting