Abstract:Graph Convolutional Networks (GCNs) show promising results for semi-supervised learning tasks on graphs, thus become favorable comparing with other approaches. Despite the remarkable success of GCNs, it is difficult to train GCNs with insufficient supervision. When labeled data are limited, the performance of GCNs becomes unsatisfying for low-degree nodes. While some prior work analyze successes and failures of GCNs on the entire model level, profiling GCNs on individual node level is still underexplored. In this paper, we analyze GCNs in regard to the node degree distribution. From empirical observation to theoretical proof, we confirm that GCNs are biased towards nodes with larger degrees with higher accuracy on them, even if high-degree nodes are underrepresented in most graphs. We further develop a novel Self-Supervised-Learning Degree-Specific GCN (SL-DSGC) that mitigate the degree-related biases of GCNs from model and data aspects. Firstly, we propose a degree-specific GCN layer that captures both discrepancies and similarities of nodes with different degrees, which reduces the inner model-aspect biases of GCNs caused by sharing the same parameters with all nodes. Secondly, we design a self-supervised-learning algorithm that creates pseudo labels with uncertainty scores on unlabeled nodes with a Bayesian neural network. Pseudo labels increase the chance of connecting to labeled neighbors for low-degree nodes, thus reducing the biases of GCNs from the data perspective. Uncertainty scores are further exploited to weight pseudo labels dynamically in the stochastic gradient descent for SL-DSGC. Experiments on three benchmark datasets show SL-DSGC not only outperforms state-of-the-art self-training/self-supervised-learning GCN methods, but also improves GCN accuracy dramatically for low-degree nodes.

An Investigation of the Impact of Normalization Schemes on GCN Modelling

Motif-induced Graph Normalization

Learning graph normalization for graph neural networks

Understanding and Resolving Performance Degradation in Graph Convolutional Networks

Normalized Activation Function: Toward Better Convergence

Understanding the Effect of GCN Convolutions in Regression Tasks

Tackling Long-tailed Distribution Issue in Graph Neural Networks via Normalization

Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalization

GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training

Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes

Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks

GC4NC: A Benchmark Framework for Graph Condensation on Node Classification with New Insights

Deeper Insights into Deep Graph Convolutional Networks: Stability and Generalization

Global Minima, Recoverability Thresholds, and Higher-Order Structure in GNNS

Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs

Neighborhood Pattern Is Crucial for Graph Convolutional Networks Performing Node Classification

Graph Neural Networks Need Cluster-Normalize-Activate Modules

BiGCN: A Bi-directional Low-Pass Filtering Graph Neural Network

Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method

A novel complex-high-order graph convolutional network paradigm: ChyGCN

Stability and Generalization of lp-Regularized Stochastic Learning for GCN