Abstract:With the widespread application of Deep Learning (DL), the black-box characteristics of DL raise questions, especially in high-stake decision-making fields like autonomous driving. Consequently, there is a growing demand for research on the interpretability of DL, leading to the emergence of eXplainable Artificial Intelligence as a current research hotspot. Current research on DL interpretability primarily focuses on transparency and post-hoc interpretability. Enhancing interpretability in transparency often requires targeted modifications to the model structure, potentially compromising the model's accuracy. Conversely, improving the interpretability of DL models based on post-hoc interpretability usually does not necessitate adjustments to the model itself. To provide a fast and accurate counterfactual explanation of DL without compromising its performance, this paper proposes a post-hoc interpretation method called relevance inference based on direct contribution to employ counterfactual reasoning in DL. In this method, direct contribution is first designed by improving Layer-wise Relevance Propagation to measure the relevance between the outputs and the inputs. Subsequently, we produce counterfactual examples based on direct contribution. Ultimately, counterfactual results for the DL model are obtained with these counterfactual examples. These counterfactual results effectively describe the behavioral boundaries of the model, facilitating a better understanding of its behavior. Additionally, direct contribution offers an easily implementable interpretable analysis method for studying model behavior. Experiments conducted on various datasets demonstrate that relevance inference can be more efficiently and accurately generate counterfactual examples compared to the state-of-the-art methods, aiding in the analysis of behavioral boundaries in intelligent decision-making models for vehicles.

Multicriteria interpretability driven deep learning

Towards Understanding Sensitive and Decisive Patterns in Explainable AI: A Case Study of Model Interpretation in Geometric Deep Learning

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

M-Rule: an Enhanced Deep Taylor Decomposition for Multi-model Interpretability

Interpreting Deep Learning Model Using Rule-based Method

A Survey of the Interpretability Aspect of Deep Learning Models

Deep PLS: A Lightweight Deep Learning Model for Interpretable and Efficient Data Analytics

Relevance Inference Based on Direct Contribution: Counterfactual Explanation to Deep Networks for Intelligent Decision-making

Interpretability in deep learning for finance: a case study for the Heston model

Interpretable and explainable machine learning: A methods‐centric overview with concrete examples

A Responsible Machine Learning Workflow with Focus on Interpretable Models, Post-hoc Explanation, and Discrimination Testing

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

How far from automatically interpreting deep learning

Interpretability of deep learning models: A survey of results

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

Interpretable Deep Learning Models: Enhancing Transparency and Trustworthiness in Explainable AI

Explaining the Unexplained: Revealing Hidden Correlations for Better Interpretability

Interpretable Deep Convolutional Neural Networks via Meta-learning

Interpretable Low-Resource Legal Decision Making

Designing Inherently Interpretable Machine Learning Models

Interpretable Artificial Intelligence through the Lens of Feature Interaction