Abstract:Over-smoothing is a challenging problem, which degrades the performance of deep graph convolutional networks (GCNs). However, existing studies for alleviating the over-smoothing problem lack either generality or effectiveness. In this paper, we analyze the underlying issues behind the over-smoothing problem, i.e., featurediversity degeneration, gradient vanishing, andmodelweights overdecaying. Inspired by this, we propose a simple yet effective plugand-play module, SkipNode, to alleviate over-smoothing. Specifically, for each middle layer of a GCN model, SkipNode randomly (or based on node degree) selects nodes to skip the convolutional operation by directly feeding their input features to the nonlinear function. Analytically, 1) skipping the convolutional operation prevents the features from losing diversity; and 2) the "skipped" nodes enable gradients to be directly passed back, thus mitigating the gradient vanishing and model weights over-decaying issues. To demonstrate the superiority of SkipNode, we conduct extensive experiments on nine popular datasets, including both homophilic and heterophilic graphs, with different graph sizes on two typical tasks: node classification and link prediction. Specifically, 1) SkipNode has strong generalizability of being applied to various GCN-based models on different datasets and tasks; and 2) SkipNode outperforms recent state-of-the-art anti-over-smoothing plugand-play modules, i.e., DropEdge and DropNode, in different settings. Code will be made publicly available on GitHub. CCS CONCEPTS • Computing methodologies → Neural networks; Artificial intelligence; Neural networks; • Mathematics of computing → Graph algorithms. This work was done when Weigang was a research intern at JD Explore Academy. Ziyu Guan is the corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Woodstock ’18, June 03–05, 2018, Woodstock, NY © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00 https://doi.org/10.1145/1122445.1122456

Alleviating Over-Smoothing Via Graph Sparsification Based on Vertex Feature Similarity

Another Perspective of Over-Smoothing: Alleviating Semantic Over-Smoothing in Deep GNNs

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

SkipNode: on Alleviating Over-smoothing for Deep Graph Convolutional Networks.

A Survey on Oversmoothing in Graph Neural Networks

ATNPA: A Unified View of Oversmoothing Alleviation in Graph Neural Networks

Graph Sparsification via Mixture of Graphs

Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View

Tackling Oversmoothing in GNN via Graph Sparsification: A Truss-based Approach

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness

A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks

Limiting Over-Smoothing and Over-Squashing of Graph Message Passing by Deep Scattering Transforms

SparGNN: Efficient Joint Feature-Model Sparsity Exploitation in Graph Neural Network Acceleration

AGNN: Alternating Graph-Regularized Neural Networks to Alleviate Over-Smoothing

Tackling Over-Smoothing for General Graph Convolutional Networks

Oversmoothing: A Nightmare for Graph Contrastive Learning?

Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks

DWSSA: Alleviating over-smoothness for deep Graph Neural Networks

Avoiding Oversmoothing in Deep Graph Neural Networks: A Multiplicative Ergodic Analysis

Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs