Abstract:Graph neural networks (GNNs) are widely used for analyzing graph-structural data and solving graph-related tasks due to their powerful expressiveness. However, existing off-the-shelf GNN-based models usually consist of no more than three layers. Deeper GNNs usually suffer from severe performance degradation due to several issues including the infamous “over-smoothing” issue, which restricts the further development of GNNs. In this article, we investigate the over-smoothing issue in deep GNNs. We discover that over-smoothing not only results in indistinguishable embeddings of graph nodes, but also alters and even corrupts their semantic structures, dubbed semantic over-smoothing. Existing techniques, e.g., graph normalization, aim at handling the former concern, but neglect the importance of preserving the semantic structures in the spatial domain, which hinders the further improvement of model performance. To alleviate the concern, we propose a cluster-keeping sparse aggregation strategy to preserve the semantic structure of embeddings in deep GNNs (especially for spatial GNNs). Particularly, our strategy heuristically redistributes the extent of aggregations for all the nodes from layers, instead of aggregating them equally, so that it enables aggregate concise yet meaningful information for deep layers. Without any bells and whistles, it can be easily implemented as a plug-and-play structure of GNNs via weighted residual connections. Last, we analyze the over-smoothing issue on the GNNs with weighted residual structures and conduct experiments to demonstrate the performance comparable to the state-of-the-arts.

Enhancing Locally Adaptive Smoothing of Graph Neural Networks Via Laplacian Node Disagreement

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Node Dependent Local Smoothing for Scalable Graph Learning

SmoothGNN: Smoothing-aware GNN for Unsupervised Node Anomaly Detection

A Unified View on Graph Neural Networks as Graph Signal Denoising

Towards Robust Graph Neural Networks against Label Noise

Resisting Over-Smoothing in Graph Neural Networks Via Dual-Dimensional Decoupling

Disambiguated Node Classification with Graph Neural Networks

Improving Distinguishability of Class for Graph Neural Networks

Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs

Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View

NAFS: A Simple Yet Tough-to-beat Baseline for Graph Representation Learning

Deep Graph Neural Networks via Flexible Subgraph Aggregation

Graph Neural Networks Beyond Compromise Between Attribute and Topology

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

Understanding Graph Neural Networks from Graph Signal Denoising Perspectives

Another Perspective of Over-Smoothing: Alleviating Semantic Over-Smoothing in Deep GNNs

Contrastive Message Passing for Robust Graph Neural Networks with Sparse Labels

AGNN: Alternating Graph-Regularized Neural Networks to Alleviate Over-Smoothing

Graph Neural Networks Inspired by Classical Iterative Algorithms