Abstract:With the widespread application of Deep Learning (DL), the black-box characteristics of DL raise questions, especially in high-stake decision-making fields like autonomous driving. Consequently, there is a growing demand for research on the interpretability of DL, leading to the emergence of eXplainable Artificial Intelligence as a current research hotspot. Current research on DL interpretability primarily focuses on transparency and post-hoc interpretability. Enhancing interpretability in transparency often requires targeted modifications to the model structure, potentially compromising the model's accuracy. Conversely, improving the interpretability of DL models based on post-hoc interpretability usually does not necessitate adjustments to the model itself. To provide a fast and accurate counterfactual explanation of DL without compromising its performance, this paper proposes a post-hoc interpretation method called relevance inference based on direct contribution to employ counterfactual reasoning in DL. In this method, direct contribution is first designed by improving Layer-wise Relevance Propagation to measure the relevance between the outputs and the inputs. Subsequently, we produce counterfactual examples based on direct contribution. Ultimately, counterfactual results for the DL model are obtained with these counterfactual examples. These counterfactual results effectively describe the behavioral boundaries of the model, facilitating a better understanding of its behavior. Additionally, direct contribution offers an easily implementable interpretable analysis method for studying model behavior. Experiments conducted on various datasets demonstrate that relevance inference can be more efficiently and accurately generate counterfactual examples compared to the state-of-the-art methods, aiding in the analysis of behavioral boundaries in intelligent decision-making models for vehicles.

Looking deeper into interpretable deep learning in neuroimaging: a comprehensive survey

Applications of interpretable deep learning in neuroimaging: a comprehensive review

A Survey of the Interpretability Aspect of Deep Learning Models

Interpretability of deep learning models: A survey of results

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Interpretability of Machine Learning Methods Applied to Neuroimaging

Explainable AI: A review of applications to neuroimaging data

A Survey on Explainable Artificial Intelligence (XAI) Techniques for Visualizing Deep Learning Models in Medical Imaging

Explainable deep learning in healthcare: A methodological survey from an attribution view

Visual Interpretability for Deep Learning: a Survey

Comparing interpretation methods in mental state decoding analyses with deep learning models

Visual Interpretability forDeepLearning

Transparency of deep neural networks for medical image analysis: A review of interpretability methods

Relevance Inference Based on Direct Contribution: Counterfactual Explanation to Deep Networks for Intelligent Decision-making

Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data

From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?

Recent Advances in Explainable Artificial Intelligence for Magnetic Resonance Imaging

On Interpretability of Artificial Neural Networks: A Survey

Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks

A survey on the interpretability of deep learning in medical diagnosis