Abstract:Recent studies have shown that graph neural networks (GNNs) exhibit strong biases towards the node degree: they usually perform satisfactorily on high-degree nodes with rich neighbor information but struggle with low-degree nodes. Existing works tackle this problem by deriving either designated GNN architectures or training strategies specifically for low-degree nodes. Though effective, these approaches unintentionally create an artificial out-of-distribution scenario, where models mainly or even only observe low-degree nodes during the training, leading to a downgraded performance for high-degree nodes that GNNs originally perform well at. In light of this, we propose a test-time augmentation framework, namely GraphPatcher, to enhance test-time generalization of any GNNs on low-degree nodes. Specifically, GraphPatcher iteratively generates virtual nodes to patch artificially created low-degree nodes via corruptions, aiming at progressively reconstructing target GNN's predictions over a sequence of increasingly corrupted nodes. Through this scheme, GraphPatcher not only learns how to enhance low-degree nodes (when the neighborhoods are heavily corrupted) but also preserves the original superior performance of GNNs on high-degree nodes (when lightly corrupted). Additionally, GraphPatcher is model-agnostic and can also mitigate the degree bias for either self-supervised or supervised GNNs. Comprehensive experiments are conducted over seven benchmark datasets and GraphPatcher consistently enhances common GNNs' overall performance by up to 3.6% and low-degree performance by up to 6.5%, significantly outperforming state-of-the-art baselines. The source code is publicly available at <a class="link-external link-https" href="https://github.com/jumxglhf/GraphPatcher" rel="external noopener nofollow">this https URL</a>.

Tackling Long-tailed Distribution Issue in Graph Neural Networks via Normalization

Motif-induced Graph Normalization

GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training

Learning graph normalization for graph neural networks

Long-tailed Graph Neural Networks Via Graph Structure Learning for Node Classification

Gentle Normalization and Translation in Graph Neural Network for Few-shot Learning

An Investigation of the Impact of Normalization Schemes on GCN Modelling

Optimizing Long-tailed Link Prediction in Graph Neural Networks through Structure Representation Enhancement

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalization

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks

Laner-GNN: Adapting to Long-Tail Degree Distribution with Latent Neighborhood Restorers

Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs

Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs

FairNorm: Fair and Fast Graph Neural Network Training

Graph Neural Networks Need Cluster-Normalize-Activate Modules

Degree-based stratification of nodes in Graph Neural Networks

Learning to Reweight for Graph Neural Network

Graph Relearn Network: Reducing Performance Variance and Improving Prediction Accuracy of Graph Neural Networks

Learning to Reweight for Generalizable Graph Neural Network

GraphPatcher: Mitigating Degree Bias for Graph Neural Networks via Test-time Augmentation