Abstract:Over-smoothing is a challenging problem, which degrades the performance of deep graph convolutional networks (GCNs). However, existing studies for alleviating the over-smoothing problem lack either generality or effectiveness. In this paper, we analyze the underlying issues behind the over-smoothing problem, i.e., featurediversity degeneration, gradient vanishing, andmodelweights overdecaying. Inspired by this, we propose a simple yet effective plugand-play module, SkipNode, to alleviate over-smoothing. Specifically, for each middle layer of a GCN model, SkipNode randomly (or based on node degree) selects nodes to skip the convolutional operation by directly feeding their input features to the nonlinear function. Analytically, 1) skipping the convolutional operation prevents the features from losing diversity; and 2) the "skipped" nodes enable gradients to be directly passed back, thus mitigating the gradient vanishing and model weights over-decaying issues. To demonstrate the superiority of SkipNode, we conduct extensive experiments on nine popular datasets, including both homophilic and heterophilic graphs, with different graph sizes on two typical tasks: node classification and link prediction. Specifically, 1) SkipNode has strong generalizability of being applied to various GCN-based models on different datasets and tasks; and 2) SkipNode outperforms recent state-of-the-art anti-over-smoothing plugand-play modules, i.e., DropEdge and DropNode, in different settings. Code will be made publicly available on GitHub. CCS CONCEPTS • Computing methodologies → Neural networks; Artificial intelligence; Neural networks; • Mathematics of computing → Graph algorithms. This work was done when Weigang was a research intern at JD Explore Academy. Ziyu Guan is the corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Woodstock ’18, June 03–05, 2018, Woodstock, NY © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00 https://doi.org/10.1145/1122445.1122456

Limiting Over-Smoothing and Over-Squashing of Graph Message Passing by Deep Scattering Transforms

Another Perspective of Over-Smoothing: Alleviating Semantic Over-Smoothing in Deep GNNs

Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks

Tackling Graph Oversquashing by Global and Local Non-Dissipativity

Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks

Alleviating Over-Smoothing Via Graph Sparsification Based on Vertex Feature Similarity

Resisting Over-Smoothing in Graph Neural Networks Via Dual-Dimensional Decoupling

Avoiding Oversmoothing in Deep Graph Neural Networks: A Multiplicative Ergodic Analysis

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

Message Passing Assisted Scalable Distributed Link Management for Ubiquitous Network

Multi-Track Message Passing: Tackling Oversmoothing and Oversquashing in Graph Learning Via Preventing Heterophily Mixing

On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology

How does over-squashing affect the power of GNNs?

A Survey on Oversmoothing in Graph Neural Networks

Oversquashing in GNNs through the lens of information contraction and graph expansion

SkipNode: on Alleviating Over-smoothing for Deep Graph Convolutional Networks.

Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View

Self-Supervised Graph Neural Networks Via Diverse and Interactive Message Passing

Boosting Graph Neural Networks by Injecting Pooling in Message Passing

ATNPA: A Unified View of Oversmoothing Alleviation in Graph Neural Networks