Abstract:Background and objective: Despite recent performance advancements, deep learning models are not yet adopted in clinical practice on a wide scale. The intrinsic intransparency of such systems is commonly cited as one major reason for this reluctance. This has motivated methods that aim to provide explanations of model functioning. Known limitations of feature-based explanations have led to an increased interest in concept-based interpretability. Testing with Concept Activation Vectors (TCAV) employs human-understandable, abstract concepts to explain model behavior. The method has previously been applied to the medical domain in the context of electronic health records, retinal fundus images and magnetic resonance imaging. Methods: We explore the usage of TCAV for building interpretable models on physiological time series, using an example of abnormality detection in electroencephalography (EEG). For this purpose, we adopt the XceptionTime model, which is suitable for multi-channel physiological data of variable sizes. The model provides state-of-the-art performance on raw EEG data and is publically available. We propose and test several ideas regarding concept definition through metadata mining, using additional labeled EEG data and extracting interpretable signal characteristics in the form of frequencies. By including our own hospital data with analog labeling, we further evaluate the robustness of our approach. Results: The tested concepts show a TCAV score distribution that is in line with the clinical expectations, i.e. concepts known to have strong links with EEG pathologies (such as epileptiform discharges) received higher scores than the neutral concepts (e.g. sex). The scores were consistent across the applied concept generation strategies. Conclusions: TCAV has the potential to improve interpretability of deep learning applied to multi-channel signals as well as to detect possible biases in the data. Still, further work on developing the strategies for concept definition and validation on clinical physiological time series is needed to better understand how to extract clinically relevant information from the concept sensitivity scores.

Explaining deep learning for ECG analysis: Building blocks for auditing and knowledge discovery

Explaining Deep Learning for ECG Analysis: Building Blocks for Auditing and Knowledge Discovery

Exploring Interpretable AI Methods for ECG Data Classification

xECGArch: a trustworthy deep learning architecture for interpretable ECG analysis considering short-term and long-term features

Analysis of a Deep Learning Model for 12-Lead ECG Classification Reveals Learned Features Similar to Diagnostic Criteria

Unveiling AI-ECG using Generative Counterfactual XAI Framework

Explainable Artificial Intelligence and Cardiac Imaging: Toward More Interpretable Models.

QXAI: Explainable AI Framework for Quantitative Analysis in Patient Monitoring Systems

Designing ECG Monitoring Healthcare System with Federated Transfer Learning and Explainable AI

Concept-based AI interpretability in physiological time-series data: Example of abnormality detection in electroencephalography

Advancing Cardiac Diagnostics: Exceptional Accuracy in Abnormal ECG Signal Classification with Cascading Deep Learning and Explainability Analysis

Explainable AI: A review of applications to neuroimaging data

A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

XAI4EEG: spectral and spatio-temporal explanation of deep learning-based seizure detection in EEG time series

Interpretability of a Deep Learning Model in the Application of Cardiac MRI Segmentation with an ACDC Challenge Dataset

Decoding 2.3 million ECGs: interpretable deep learning for advancing cardiovascular diagnosis and mortality risk stratification

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

Towards explainability in artificial intelligence frameworks for heartcare: A comprehensive survey

Interpretable ECG Analysis for Myocardial Infarction Detection through Counterfactuals

M-XAF: Medical Explainable Diagnosis System of Atrial Fibrillation Based on Medical Knowledge and Semantic Representation Fusion

BenchXAI: Comprehensive Benchmarking of Post-hoc Explainable AI Methods on Multi-Modal Biomedical Data