Abstract:Graph Neural Networks (GNNs) are proposed without considering the agnostic distribution shifts between training graphs and testing graphs, inducing the degeneration of the generalization ability of GNNs in Out-Of-Distribution (OOD) settings. The fundamental reason for such degeneration is that most GNNs are developed based on the I.I.D hypothesis. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. This learning mechanism inherits from the common characteristics of machine learning approaches. However, such spurious correlations may change in the wild testing environments, leading to the failure of GNNs. Therefore, eliminating the impact of spurious correlations is crucial for stable GNN models. To this end, in this paper, we argue that the spurious correlation exists among subgraph-level units and analyze the degeneration of GNN in causal view. Based on the causal view analysis, we propose a general causal representation framework for stable GNN, called StableGNN. The main idea of this framework is to extract high-level representations from raw graph data first and resort to the distinguishing ability of causal inference to help the model get rid of spurious correlations. Particularly, to extract meaningful high-level representations, we exploit a differentiable graph pooling layer to extract subgraph-based representations by an end-to-end manner. Furthermore, inspired by the confounder balancing techniques from causal inference, based on the learned high-level representations, we propose a causal variable distinguishing regularizer to correct the biased training distribution by learning a set of sample weights. Hence, GNNs would concentrate more on the true connection between discriminative substructures and labels. Extensive experiments are conducted on both synthetic datasets with various distribution shift degrees and eight real-world OOD graph datasets. The results well verify that the proposed model StableGNN not only outperforms the state-of-the-arts but also provides a flexible framework to enhance existing GNNs. In addition, the interpretability experiments validate that StableGNN could leverage causal structures for predictions.

Fortune Favors the Invariant: Enhancing GNNs’ Generalizability with Invariant Graph Learning

Invariant Factor Graph Neural Networks

Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization

Graph Invariant Learning with Subgraph Co-mixup for Out-Of-Distribution Generalization

IENE: Identifying and Extrapolating the Node Environment for Out-of-Distribution Generalization on Graphs

Utilizing Edge Features in Graph Neural Networks Via Variational Information Maximization

Learning Invariant Representations of Graph Neural Networks via Cluster Generalization

Does Invariant Graph Learning via Environment Augmentation Learn Invariance?

A Simple Data Augmentation for Graph Classification: A Perspective of Equivariance and Invariance

Brave the Wind and the Waves: Discovering Robust and Generalizable Graph Lottery Tickets.

Unifying Invariant and Variant Features for Graph Out-of-Distribution via Probability of Necessity and Sufficiency

Generalizing Graph Neural Networks on Out-of-Distribution Graphs

FLOOD: A Flexible Invariant Learning Framework for Out-of-Distribution Generalization on Graphs

Handling Distribution Shifts on Graphs: An Invariance Perspective

Leveraging Invariant Principle for Heterophilic Graph Structure Distribution Shifts

Subgraph Aggregation for Out-of-Distribution Generalization on Graphs

GSINA: Improving Subgraph Extraction for Graph Invariant Learning via Graph Sinkhorn Attention

Dissecting the Failure of Invariant Learning on Graphs

Scale Invariance of Graph Neural Networks

Graph Out-of-Distribution Generalization with Controllable Data Augmentation

One Fits All: Learning Fair Graph Neural Networks for Various Sensitive Attributes