Abstract:In graph classification, attention- and pooling-based graph neural networks (GNNs) predominate to extract salient features from the input graph and support the prediction. They mostly follow the paradigm of “learning to attend”, which maximizes the mutual information between the attended graph and the ground-truth label. However, this paradigm causes GNN classifiers to indiscriminately absorb all statistical correlations between input features and labels in the training data, without distinguishing the causal and noncausal effects of features. Rather than emphasizing causal features, the attended graphs tend to rely on noncausal features as shortcuts to predictions. These shortcut features may easily change outside the training distribution, thereby leading to poor generalization for GNN classifiers. In this paper, we take a causal view on GNN modeling. Under our causal assumption, the shortcut feature serves as a confounder between the causal feature and prediction. It misleads the classifier into learning spurious correlations that facilitate prediction in in-distribution (ID) test evaluation, while causing significant performance drop in out-of-distribution (OOD) test data. To address this issue, we employ the backdoor adjustment from causal theory — combining each causal feature with various shortcut features, to identify causal patterns and mitigate the confounding effect. Specifically, we employ attention modules to estimate the causal and shortcut features of the input graph. Then, a memory bank collects the estimated shortcut features, enhancing the diversity of shortcut features for combination. Simultaneously, we apply the prototype strategy to improve the consistency of intra-class causal features. We term our method as CAL+, which can promote stable relationships between causal estimation and prediction, regardless of distribution changes. Extensive experiments on synthetic and real-world OOD benchmarks demonstrate our method’s effectiveness in improving OOD generalization. Our codes are released at https://github.com/shuyao-wang/CAL-plus.

Learning 1D Causal Visual Representation with De-focus Attention Networks

Learning Stereoscopic Visual Attention Model for 3d Video

Causal Attention for Unbiased Visual Recognition

Deep Co-Attention Network for Multi-View Subspace Learning

Interpreting Low-level Vision Models with Causal Effect Maps

Causal Image Modeling for Efficient Visual Understanding

Vision-and-Language Navigation via Causal Learning

Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification

Learning Hierarchal Channel Attention for Fine-grained Visual Classification.

Causal Attention for Vision-Language Tasks

Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention

Hierarchical Causality Network: Find the Effective Connectivity in Visual Cortex

TL-CCL: Two-level causal contrastive learning for multi-label ocular disease diagnosis with fundus images

FAN: Focused Attention Networks

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

Causality-based Cross-Modal Representation Learning for Vision-and-Language Navigation

Causality Compensated Attention for Contextual Biased Visual Recognition

Learning high-level concepts by training a deep network on eye fixations

Dependent Multi-Task Learning with Causal Intervention for Image Captioning.

A Generic Visualization Approach for Convolutional Neural Networks

Enhancing Out-of-distribution Generalization on Graphs via Causal Attention Learning