Abstract:Graph Neural Networks (GNNs) are proposed without considering the agnostic distribution shifts between training graphs and testing graphs, inducing the degeneration of the generalization ability of GNNs in Out-Of-Distribution (OOD) settings. The fundamental reason for such degeneration is that most GNNs are developed based on the I.I.D hypothesis. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. This learning mechanism inherits from the common characteristics of machine learning approaches. However, such spurious correlations may change in the wild testing environments, leading to the failure of GNNs. Therefore, eliminating the impact of spurious correlations is crucial for stable GNN models. To this end, in this paper, we argue that the spurious correlation exists among subgraph-level units and analyze the degeneration of GNN in causal view. Based on the causal view analysis, we propose a general causal representation framework for stable GNN, called StableGNN. The main idea of this framework is to extract high-level representations from raw graph data first and resort to the distinguishing ability of causal inference to help the model get rid of spurious correlations. Particularly, to extract meaningful high-level representations, we exploit a differentiable graph pooling layer to extract subgraph-based representations by an end-to-end manner. Furthermore, inspired by the confounder balancing techniques from causal inference, based on the learned high-level representations, we propose a causal variable distinguishing regularizer to correct the biased training distribution by learning a set of sample weights. Hence, GNNs would concentrate more on the true connection between discriminative substructures and labels. Extensive experiments are conducted on both synthetic datasets with various distribution shift degrees and eight real-world OOD graph datasets. The results well verify that the proposed model StableGNN not only outperforms the state-of-the-arts but also provides a flexible framework to enhance existing GNNs. In addition, the interpretability experiments validate that StableGNN could leverage causal structures for predictions.

Out-of-Distribution Generalization with Causal Feature Separation

Discovering causally invariant features for out-of-distribution generalization

Learning Causal Semantic Representation for Out-of-Distribution Prediction

Graph Out-of-Distribution Generalization via Causal Intervention

Invariant Feature Learning Based on Causal Inference from Heterogeneous Environments

Enhancing Out-of-distribution Generalization on Graphs via Causal Attention Learning

Out-of-Distribution Generalization Via Style and Spuriousness Eliminating

Invariant Learning via Probability of Sufficient and Necessary Causes

A Causality-Aware Perspective on Domain Generalization via Domain Intervention

Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?

Revisiting Spurious Correlation in Domain Generalization

Towards a Theoretical Framework of Out-of-Distribution Generalization

Out-of-distribution Generalization via Partial Feature Decorrelation

Generalizing Graph Neural Networks on Out-of-Distribution Graphs

A step towards the applicability of algorithms based on invariant causal learning on observational data

Few-Shot Causal Representation Learning for Out-of-Distribution Generalization on Heterogeneous Graphs

Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization

CASE: Exploiting Intra-class Compactness and Inter-class Separability of Feature Embeddings for Out-of-Distribution Detection

Causal Representation Learning for Out-of-Distribution Recommendation

Unraveling the Key Components of OOD Generalization via Diversification

Multi-Instance Causal Representation Learning for Instance Label Prediction and Out-of-Distribution Generalization