Abstract:Over-smoothing is a challenging problem, which degrades the performance of deep graph convolutional networks (GCNs). However, existing studies for alleviating the over-smoothing problem lack either generality or effectiveness. In this paper, we analyze the underlying issues behind the over-smoothing problem, i.e., featurediversity degeneration, gradient vanishing, andmodelweights overdecaying. Inspired by this, we propose a simple yet effective plugand-play module, SkipNode, to alleviate over-smoothing. Specifically, for each middle layer of a GCN model, SkipNode randomly (or based on node degree) selects nodes to skip the convolutional operation by directly feeding their input features to the nonlinear function. Analytically, 1) skipping the convolutional operation prevents the features from losing diversity; and 2) the "skipped" nodes enable gradients to be directly passed back, thus mitigating the gradient vanishing and model weights over-decaying issues. To demonstrate the superiority of SkipNode, we conduct extensive experiments on nine popular datasets, including both homophilic and heterophilic graphs, with different graph sizes on two typical tasks: node classification and link prediction. Specifically, 1) SkipNode has strong generalizability of being applied to various GCN-based models on different datasets and tasks; and 2) SkipNode outperforms recent state-of-the-art anti-over-smoothing plugand-play modules, i.e., DropEdge and DropNode, in different settings. Code will be made publicly available on GitHub. CCS CONCEPTS • Computing methodologies → Neural networks; Artificial intelligence; Neural networks; • Mathematics of computing → Graph algorithms. This work was done when Weigang was a research intern at JD Explore Academy. Ziyu Guan is the corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Woodstock ’18, June 03–05, 2018, Woodstock, NY © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00 https://doi.org/10.1145/1122445.1122456

Revisiting Over-smoothing in Deep GCNs

Tackling Over-Smoothing for General Graph Convolutional Networks

Preventing Over-Smoothing for Hypergraph Neural Networks

Demystifying Oversmoothing in Attention-Based Graph Neural Networks

A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks

SkipNode: on Alleviating Over-smoothing for Deep Graph Convolutional Networks.

Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks

SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networks

Oversmoothing: A Nightmare for Graph Contrastive Learning?

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning

AGNN: Alternating Graph-Regularized Neural Networks to Alleviate Over-Smoothing

Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks

A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Neural Networks

Learning to Control the Smoothness of Graph Convolutional Network Features

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Graph Neural Networks Do Not Always Oversmooth

DRGCN: Dynamic Evolving Initial Residual for Deep Graph Convolutional Networks

A Survey on Oversmoothing in Graph Neural Networks

Simple and Deep Graph Convolutional Networks

ATNPA: A Unified View of Oversmoothing Alleviation in Graph Neural Networks