Abstract:With the widespread application of Deep Learning (DL), the black-box characteristics of DL raise questions, especially in high-stake decision-making fields like autonomous driving. Consequently, there is a growing demand for research on the interpretability of DL, leading to the emergence of eXplainable Artificial Intelligence as a current research hotspot. Current research on DL interpretability primarily focuses on transparency and post-hoc interpretability. Enhancing interpretability in transparency often requires targeted modifications to the model structure, potentially compromising the model's accuracy. Conversely, improving the interpretability of DL models based on post-hoc interpretability usually does not necessitate adjustments to the model itself. To provide a fast and accurate counterfactual explanation of DL without compromising its performance, this paper proposes a post-hoc interpretation method called relevance inference based on direct contribution to employ counterfactual reasoning in DL. In this method, direct contribution is first designed by improving Layer-wise Relevance Propagation to measure the relevance between the outputs and the inputs. Subsequently, we produce counterfactual examples based on direct contribution. Ultimately, counterfactual results for the DL model are obtained with these counterfactual examples. These counterfactual results effectively describe the behavioral boundaries of the model, facilitating a better understanding of its behavior. Additionally, direct contribution offers an easily implementable interpretable analysis method for studying model behavior. Experiments conducted on various datasets demonstrate that relevance inference can be more efficiently and accurately generate counterfactual examples compared to the state-of-the-art methods, aiding in the analysis of behavioral boundaries in intelligent decision-making models for vehicles.

Transparency and Explanation in Deep Reinforcement Learning Neural Networks

Verbal Explanations for Deep Reinforcement Learning Neural Networks with Attention on Extracted Features.

Explainable Deep Reinforcement Learning: State of the Art and Challenges

A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges

Interpretable Deep Learning Models: Enhancing Transparency and Trustworthiness in Explainable AI

How Do You Act? An Empirical Study to Understand Behavior of Deep Reinforcement Learning Agents

Learning Symbolic Rules for Interpretable Deep Reinforcement Learning

Relevance Inference Based on Direct Contribution: Counterfactual Explanation to Deep Networks for Intelligent Decision-making

Explainable Deep Learning: A Visual Analytics Approach with Transition Matrices

Integrated Commonsense Reasoning and Deep Learning for Transparent Decision Making in Robotics

Neural Reasoning Networks: Efficient Interpretable Neural Networks With Automatic Textual Explanations

Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems

Increasing Transparency of Reinforcement Learning using Shielding for Human Preferences and Explanations

Improving Transparency of Deep Neural Inference Process

Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System

Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks

Neural network interpretability with layer-wise relevance propagation: novel techniques for neuron selection and visualization

A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts

Bridging Trustworthiness and Open-World Learning: An Exploratory Neural Approach for Enhancing Interpretability, Generalization, and Robustness

Reason induced visual attention for explainable autonomous driving

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models