Abstract:Most graph neural networks (GNNs) are proposed and evaluated under independent and identically distributed (IID) training and testing data. In real-world applications, however, agnostic distribution shifts from training to testing naturally exist, leading to unstable prediction of traditional GNNs. To bridge the gap, we pursue stable prediction on graphs, i.e., to achieve high average performance and low performance variance (stability) across non-IID testing graphs. The key to stable prediction lies in capturing stable properties that are resilient to distribution shifts. In this light, we aim to identify neighbor nodes (properties) in neighborhood aggregation that are consistently important for prediction under heterogeneous distribution shifts. To achieve this target, we propose a model-agnostic stable learning framework for GNNs. The framework performs biased selection on the observed training graph, resulting in multiple non-IID graph subsets. We train one weight predictor per subset to measure the importance of properties under a particular distribution shift, and multiple predictors could tell the properties that are consistently important. An important property should contribute to high average performance and also stability (low performance variance) across non-IID subsets. In this regard, in training importance predictors, we introduce a globally stable regularizer to reduce the variance of training losses across non-IID graph datasets. Based on the importance weights of properties across nonIID subsets, a locally stable regularizer down-weights unstable properties in prediction. We conduct extensive experiments on several graph benchmarks and a noisy industrial recommendation dataset where distribution shifts exist. The results demonstrate that our method outperforms various state-of-the-art GNNs for stable prediction on graphs with agnostic distribution shifts.

Learning Stable Graphs from Multiple Environments with Selection Bias

Homophily-Enhanced Self-Supervision for Graph Structure Learning: Insights and Directions.

Social Learning in Community Structured Graphs

Joint learning of graph and latent representation for unsupervised feature selection

Stable Prediction Across Unknown Environments

Graph Learning Across Data Silos

Learning safe graph construction from multiple graphs

A Graph-Assisted Framework for Multiple Graph Learning

Stable Prediction on Graphs with Agnostic Distribution Shift

Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalization

Self-adjusted graph based semi-supervised embedded feature selection

Does Invariant Graph Learning via Environment Augmentation Learn Invariance?

Efficient and Stable Unsupervised Feature Selection Based on Novel Structured Graph and Data Discrepancy Learning

Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminating

A Unifying Causal Framework for Analyzing Dataset Shift-stable Learning Algorithms

Diversified Multiscale Graph Learning with Graph Self-Correction

Stable Prediction on Graphs with Agnostic Distribution Shifts

Environment-Invariant Curriculum Relation Learning for Fine-Grained Scene Graph Generation

Continual Graph Learning: A Survey

A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions

Unsupervised Discriminative Feature Selection via Contrastive Graph Learning