Abstract:Graph Neural Networks (GNNs) are proposed without considering the agnostic distribution shifts between training graphs and testing graphs, inducing the degeneration of the generalization ability of GNNs in Out-Of-Distribution (OOD) settings. The fundamental reason for such degeneration is that most GNNs are developed based on the I.I.D hypothesis. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. This learning mechanism inherits from the common characteristics of machine learning approaches. However, such spurious correlations may change in the wild testing environments, leading to the failure of GNNs. Therefore, eliminating the impact of spurious correlations is crucial for stable GNN models. To this end, in this paper, we argue that the spurious correlation exists among subgraph-level units and analyze the degeneration of GNN in causal view. Based on the causal view analysis, we propose a general causal representation framework for stable GNN, called StableGNN. The main idea of this framework is to extract high-level representations from raw graph data first and resort to the distinguishing ability of causal inference to help the model get rid of spurious correlations. Particularly, to extract meaningful high-level representations, we exploit a differentiable graph pooling layer to extract subgraph-based representations by an end-to-end manner. Furthermore, inspired by the confounder balancing techniques from causal inference, based on the learned high-level representations, we propose a causal variable distinguishing regularizer to correct the biased training distribution by learning a set of sample weights. Hence, GNNs would concentrate more on the true connection between discriminative substructures and labels. Extensive experiments are conducted on both synthetic datasets with various distribution shift degrees and eight real-world OOD graph datasets. The results well verify that the proposed model StableGNN not only outperforms the state-of-the-arts but also provides a flexible framework to enhance existing GNNs. In addition, the interpretability experiments validate that StableGNN could leverage causal structures for predictions.

Invariant Factor Graph Neural Networks

Fortune Favors the Invariant: Enhancing GNNs’ Generalizability with Invariant Graph Learning

Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization

Generalizing Graph Neural Networks on Out-of-Distribution Graphs

OOD-GNN: Out-of-Distribution Generalized Graph Neural Network

Graph Out-of-Distribution Generalization via Causal Intervention

Invariant Graph Learning for Causal Effect Estimation

Graph Invariant Learning with Subgraph Co-mixup for Out-Of-Distribution Generalization

Utilizing Edge Features in Graph Neural Networks Via Variational Information Maximization

Factor Graph Neural Networks

Discovering Invariant Rationales for Graph Neural Networks

IENE: Identifying and Extrapolating the Node Environment for Out-of-Distribution Generalization on Graphs

Learning Invariant Representations of Graph Neural Networks via Cluster Generalization

Graph Information Vanishing Phenomenon in Implicit Graph Neural Networks

FLOOD: A Flexible Invariant Learning Framework for Out-of-Distribution Generalization on Graphs

Isometric Graph Neural Networks

Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs

Learning to Reweight for Graph Neural Network

Scale Invariance of Graph Neural Networks

Invariant Graph Learning for Treatment Effect Estimation from Networked Observational Data

Out-of-Distribution Generalized Dynamic Graph Neural Network with Disentangled Intervention and Invariance Promotion