Abstract:Graph Neural Networks (GNNs) are proposed without considering the agnostic distribution shifts between training graphs and testing graphs, inducing the degeneration of the generalization ability of GNNs in Out-Of-Distribution (OOD) settings. The fundamental reason for such degeneration is that most GNNs are developed based on the I.I.D hypothesis. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. This learning mechanism inherits from the common characteristics of machine learning approaches. However, such spurious correlations may change in the wild testing environments, leading to the failure of GNNs. Therefore, eliminating the impact of spurious correlations is crucial for stable GNN models. To this end, in this paper, we argue that the spurious correlation exists among subgraph-level units and analyze the degeneration of GNN in causal view. Based on the causal view analysis, we propose a general causal representation framework for stable GNN, called StableGNN. The main idea of this framework is to extract high-level representations from raw graph data first and resort to the distinguishing ability of causal inference to help the model get rid of spurious correlations. Particularly, to extract meaningful high-level representations, we exploit a differentiable graph pooling layer to extract subgraph-based representations by an end-to-end manner. Furthermore, inspired by the confounder balancing techniques from causal inference, based on the learned high-level representations, we propose a causal variable distinguishing regularizer to correct the biased training distribution by learning a set of sample weights. Hence, GNNs would concentrate more on the true connection between discriminative substructures and labels. Extensive experiments are conducted on both synthetic datasets with various distribution shift degrees and eight real-world OOD graph datasets. The results well verify that the proposed model StableGNN not only outperforms the state-of-the-arts but also provides a flexible framework to enhance existing GNNs. In addition, the interpretability experiments validate that StableGNN could leverage causal structures for predictions.

Learning Invariant Representations of Graph Neural Networks via Cluster Generalization

Fortune Favors the Invariant: Enhancing GNNs’ Generalizability with Invariant Graph Learning

Invariant Factor Graph Neural Networks

Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization

AdaGCL

Graph Invariant Learning with Subgraph Co-mixup for Out-Of-Distribution Generalization

CAGNN: Cluster-Aware Graph Neural Networks for Unsupervised Graph Representation Learning

Towards Understanding the Generalization of Graph Neural Networks.

Local Clustering Graph Neural Networks

Generalizing Graph Neural Networks on Out-of-Distribution Graphs

Synergistic Deep Graph Clustering Network

Scale Invariance of Graph Neural Networks

GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks

Universal Inceptive GNNs by Eliminating the Smoothness-generalization Dilemma

Towards Understanding Generalization of Graph Neural Networks.

ECGN: A Cluster-Aware Approach to Graph Neural Networks for Imbalanced Classification

Graph Out-of-Distribution Generalization via Causal Intervention

ScaleNet: Scale Invariance Learning in Directed Graphs

Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs

Graph Neural Network with Curriculum Learning for Imbalanced Node Classification

Pre-Train and Learn: Preserving Global Information for Graph Neural Networks.