Abstract:In graph classification, attention- and pooling-based graph neural networks (GNNs) prevail to extract the critical features from the input graph and support the prediction. They mostly follow the paradigm of "learning to attend", which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm makes GNN classifiers recklessly absorb all the statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Instead of underscoring the causal features, the attended graphs are prone to visit the noncausal features as the shortcut to predictions. Such shortcut features might easily change outside the training distribution, thereby making the GNN classifiers suffer from poor generalization. In this work, we take a causal look at the GNN modeling for graph classification. With our causal assumption, the shortcut feature serves as a confounder between the causal feature and prediction. It tricks the classifier to learn spurious correlations that facilitate the prediction in in-distribution (ID) test evaluation, while causing the performance drop in out-of-distribution (OOD) test data. To endow the classifier with better interpretation and generalization, we propose the Causal Attention Learning (CAL) strategy, which discovers the causal patterns and mitigates the confounding effect of shortcuts. Specifically, we employ attention modules to estimate the causal and shortcut features of the input graph. We then parameterize the backdoor adjustment of causal theory - combine each causal feature with various shortcut features. It encourages the stable relationships between the causal estimation and the prediction, regardless of the changes in shortcut parts and distributions. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness of CAL.

Understanding Attention and Generalization in Graph Neural Networks

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Revisiting Attention-Based Graph Neural Networks for Graph Classification

Global-local graph attention: unifying global and local attention for node classification

How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

Towards Deep Attention in Graph Neural Networks: Problems and Remedies

Improving Attention Mechanism in Graph Neural Networks via Cardinality Preservation

How Attentive are Graph Attention Networks?

Attention-based graph neural networks: a survey

Graph Ordering Attention Networks

Bi-Level Attention Graph Neural Networks

Graph Oriented Attention Networks

Causal Attention for Interpretable and Generalizable Graph Classification

High-order graph attention network

A Regularized Attention Mechanism for Graph Attention Networks

Not All Neighbors Are Worth Attending To: Graph Selective Attention Networks for Semi-supervised Learning

An end-to-end attention-based approach for learning on graphs

A Bird's-Eye Tutorial of Graph Attention Architectures

FairGAT: Fairness-aware Graph Attention Networks

Deep Graph Attention Model

Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism