Abstract:In graph classification, attention- and pooling-based graph neural networks (GNNs) prevail to extract the critical features from the input graph and support the prediction. They mostly follow the paradigm of "learning to attend", which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm makes GNN classifiers recklessly absorb all the statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Instead of underscoring the causal features, the attended graphs are prone to visit the noncausal features as the shortcut to predictions. Such shortcut features might easily change outside the training distribution, thereby making the GNN classifiers suffer from poor generalization. In this work, we take a causal look at the GNN modeling for graph classification. With our causal assumption, the shortcut feature serves as a confounder between the causal feature and prediction. It tricks the classifier to learn spurious correlations that facilitate the prediction in in-distribution (ID) test evaluation, while causing the performance drop in out-of-distribution (OOD) test data. To endow the classifier with better interpretation and generalization, we propose the Causal Attention Learning (CAL) strategy, which discovers the causal patterns and mitigates the confounding effect of shortcuts. Specifically, we employ attention modules to estimate the causal and shortcut features of the input graph. We then parameterize the backdoor adjustment of causal theory - combine each causal feature with various shortcut features. It encourages the stable relationships between the causal estimation and the prediction, regardless of the changes in shortcut parts and distributions. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness of CAL.

G-Inspector: Recurrent Attention Model on Graph.

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Revisiting Attention-Based Graph Neural Networks for Graph Classification

Deep Graph Attention Model

Attention Models in Graphs: A Survey

High-order graph attention network

Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism

Understanding Attention and Generalization in Graph Neural Networks

A Regularized Attention Mechanism for Graph Attention Networks

Attention-based graph neural networks: a survey

Causal Attention for Interpretable and Generalizable Graph Classification

Attention-based Graph Neural Network for Semi-supervised Learning

On Classification Thresholds for Graph Attention with Edge Features

AddGraph: Anomaly Detection in Dynamic Graph Using Attention-based Temporal GCN

Graph-based Solutions with Residuals for Intrusion Detection: the Modified E-GraphSAGE and E-ResGAT Algorithms

Node Classification via Semantic-Structural Attention-Enhanced Graph Convolutional Networks

CrimeGAT: Leveraging Graph Attention Networks for Enhanced Predictive Policing in Criminal Networks

Discovering latent node Information by graph attention network

Global-local graph attention: unifying global and local attention for node classification

GRAM: An Interpretable Approach for Graph Anomaly Detection using Gradient Attention Maps

Hierarchical Graph Attention Networks for Semi-Supervised Node Classification