Abstract:Explainability is crucial for probing graph neural networks (GNNs), answering questions like “Why the GNN model makes a certain prediction?”. Feature attribution is a prevalent technique of highlighting the explanatory subgraph in the input graph, which plausibly leads the GNN model to make its prediction. Various attribution methods have been proposed to exploit gradient-like or attention scores as the attributions of edges, then select the salient edges with top attribution scores as the explanation. However, most of these works make an untenable assumption — the selected edges are linearly independent — thus leaving the dependencies among edges largely unexplored, especially their coalition effect. We demonstrate unambiguous drawbacks of this assumption — making the explanatory subgraph unfaithful and verbose. To address this challenge, we propose a reinforcement learning agent, Reinforced Causal Explainer (RC-Explainer). It frames the explanation task as a sequential decision process — an explanatory subgraph is successively constructed by adding a salient edge to connect the previously selected subgraph. Technically, its policy network predicts the action of edge addition, and gets a reward that quantifies the action’s causal effect on the prediction. Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations. It is trained via policy gradient to optimize the reward stream of edge sequences. As such, RC-Explainer is able to generate faithful and concise explanations, and has a better generalization power to unseen graphs. When explaining different GNNs on three graph classification datasets, RC-Explainer achieves better or comparable performance to state-of-the-art approaches w.r.t. two quantitative metrics: predictive accuracy, contrastivity, and safely passes sanity checks and visual inspections. Codes and datasets are available at https://github.com/xiangwang1223/reinforced_causal_explainer .

Keep the Faith: Faithful Explanations in Convolutional Neural Networks for Case-Based Reasoning

A Pixel-Level Explainable Approach of Convolutional Neural Networks and Its Application

Deep Learning for Case-Based Reasoning Through Prototypes: A Neural Network That Explains Its Predictions

HQProtoPNet: an Evidence-Based Model for Interpretable Image Recognition.

Faithful and Consistent Graph Neural Network Explanations with Rationale Alignment

Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluation

Faithful and Plausible Natural Language Explanations for Image Classification: A Pipeline Approach

This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagation

Evaluation and Improvement of Interpretability for Self-Explainable Part-Prototype Networks

ProtGNN: Towards Self-Explaining Graph Neural Networks.

This Looks Like That: Deep Learning for Interpretable Image Recognition

Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces

Enhanced Prototypical Part Network (EPPNet) For Explainable Image Classification Via Prototypes

Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes

ProtoArgNet: Interpretable Image Classification with Super-Prototypes and Argumentation [Technical Report]

Faithfulness Tests for Natural Language Explanations

Visual correspondence-based explanations improve AI robustness and human-AI team accuracy

Towards Faithful and Consistent Explanations for Graph Neural Networks

Improving Prototypical Visual Explanations with Reward Reweighing, Reselection, and Retraining

Reinforced Causal Explainer for Graph Neural Networks.