Abstract:Explainability is crucial for probing graph neural networks (GNNs), answering questions like “Why the GNN model makes a certain prediction?”. Feature attribution is a prevalent technique of highlighting the explanatory subgraph in the input graph, which plausibly leads the GNN model to make its prediction. Various attribution methods have been proposed to exploit gradient-like or attention scores as the attributions of edges, then select the salient edges with top attribution scores as the explanation. However, most of these works make an untenable assumption — the selected edges are linearly independent — thus leaving the dependencies among edges largely unexplored, especially their coalition effect. We demonstrate unambiguous drawbacks of this assumption — making the explanatory subgraph unfaithful and verbose. To address this challenge, we propose a reinforcement learning agent, Reinforced Causal Explainer (RC-Explainer). It frames the explanation task as a sequential decision process — an explanatory subgraph is successively constructed by adding a salient edge to connect the previously selected subgraph. Technically, its policy network predicts the action of edge addition, and gets a reward that quantifies the action’s causal effect on the prediction. Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations. It is trained via policy gradient to optimize the reward stream of edge sequences. As such, RC-Explainer is able to generate faithful and concise explanations, and has a better generalization power to unseen graphs. When explaining different GNNs on three graph classification datasets, RC-Explainer achieves better or comparable performance to state-of-the-art approaches w.r.t. two quantitative metrics: predictive accuracy, contrastivity, and safely passes sanity checks and visual inspections. Codes and datasets are available at https://github.com/xiangwang1223/reinforced_causal_explainer .

A Simplified Benchmark for Ambiguous Explanations of Knowledge Graph Link Prediction Using Relational Graph Convolutional Networks (Student Abstract)

LinkLogic: A New Method and Benchmark for Explainable Knowledge Graph Predictions

KGExplainer: Towards Exploring Connected Subgraph Explanations for Knowledge Graph Completion

Verifying Relational Explanations: A Probabilistic Approach

Evaluating Link Prediction Explanations for Graph Neural Networks

Self-Explainable Graph Neural Networks for Link Prediction

KGEx: Explaining Knowledge Graph Embeddings via Subgraph Sampling and Knowledge Distillation

Path-based Explanation for Knowledge Graph Completion

KR-GCN: Knowledge-aware Reasoning with Graph Convolution Network for Explainable Recommendation

Effects of Locality and Rule Language on Explanations for Knowledge Graph Embeddings

Reinforced Causal Explainer for Graph Neural Networks.

Explainable Sparse Knowledge Graph Completion Via High-order Graph Reasoning Network

Towards explaining graph neural networks via preserving prediction ranking and structural dependency

Generating Explanations for Conceptual Validation of Graph Neural Networks: An Investigation of Symbolic Predicates Learned on Relevance-Ranked Sub-Graphs

KE-X: Towards Subgraph Explanations of Knowledge Graph Embedding Based on Knowledge Information Gain

Towards Faithful and Consistent Explanations for Graph Neural Networks

Explainable Link Prediction in Knowledge Hypergraphs.

GradXKG: A Universal Explain-per-use Temporal Knowledge Graph Explainer

RegExplainer: Generating Explanations for Graph Neural Networks in Regression Tasks

Robust Counterfactual Explanations on Graph Neural Networks.

On Explainability of Graph Neural Networks Via Subgraph Explorations.