Abstract:Over-smoothing is a challenging problem, which degrades the performance of deep graph convolutional networks (GCNs). However, existing studies for alleviating the over-smoothing problem lack either generality or effectiveness. In this paper, we analyze the underlying issues behind the over-smoothing problem, i.e., featurediversity degeneration, gradient vanishing, andmodelweights overdecaying. Inspired by this, we propose a simple yet effective plugand-play module, SkipNode, to alleviate over-smoothing. Specifically, for each middle layer of a GCN model, SkipNode randomly (or based on node degree) selects nodes to skip the convolutional operation by directly feeding their input features to the nonlinear function. Analytically, 1) skipping the convolutional operation prevents the features from losing diversity; and 2) the "skipped" nodes enable gradients to be directly passed back, thus mitigating the gradient vanishing and model weights over-decaying issues. To demonstrate the superiority of SkipNode, we conduct extensive experiments on nine popular datasets, including both homophilic and heterophilic graphs, with different graph sizes on two typical tasks: node classification and link prediction. Specifically, 1) SkipNode has strong generalizability of being applied to various GCN-based models on different datasets and tasks; and 2) SkipNode outperforms recent state-of-the-art anti-over-smoothing plugand-play modules, i.e., DropEdge and DropNode, in different settings. Code will be made publicly available on GitHub. CCS CONCEPTS • Computing methodologies → Neural networks; Artificial intelligence; Neural networks; • Mathematics of computing → Graph algorithms. This work was done when Weigang was a research intern at JD Explore Academy. Ziyu Guan is the corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Woodstock ’18, June 03–05, 2018, Woodstock, NY © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00 https://doi.org/10.1145/1122445.1122456

CurvDrop: A Ricci Curvature Based Approach to Prevent Graph Neural Networks from Over-Smoothing and Over-Squashing.

How Curvature Enhance the Adaptation Power of Framelet GCNs

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Understanding over-squashing and bottlenecks on graphs via curvature

Revisiting Over-smoothing and Over-squashing using Ollivier's Ricci Curvature

Understanding Dropout for Graph Neural Networks

DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

Ricci Curvature-Based Semi-Supervised Learning on an Attributed Network

Adaptive curvature exploration geometric graph neural network

SkipNode: on Alleviating Over-smoothing for Deep Graph Convolutional Networks.

FlexiDrop: Theoretical Insights and Practical Advances in Random Dropout Method on GNNs

Hyperbolic Curvature Graph Neural Network

Analyzing Neural Network Robustness Using Graph Curvature

Graph Dilated Network with Rejection Mechanism

The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisited

Graph Coarsening with Neural Networks

Revealing Decurve Flows for Generalized Graph Propagation

Scaling Up Graph Neural Networks Via Graph Coarsening

Deeper Exploiting Graph Structure Information by Discrete Ricci Curvature in a Graph Transformer.

Deep Graph Neural Networks via Flexible Subgraph Aggregation