Abstract:Over-smoothing is a challenging problem, which degrades the performance of deep graph convolutional networks (GCNs). However, existing studies for alleviating the over-smoothing problem lack either generality or effectiveness. In this paper, we analyze the underlying issues behind the over-smoothing problem, i.e., featurediversity degeneration, gradient vanishing, andmodelweights overdecaying. Inspired by this, we propose a simple yet effective plugand-play module, SkipNode, to alleviate over-smoothing. Specifically, for each middle layer of a GCN model, SkipNode randomly (or based on node degree) selects nodes to skip the convolutional operation by directly feeding their input features to the nonlinear function. Analytically, 1) skipping the convolutional operation prevents the features from losing diversity; and 2) the "skipped" nodes enable gradients to be directly passed back, thus mitigating the gradient vanishing and model weights over-decaying issues. To demonstrate the superiority of SkipNode, we conduct extensive experiments on nine popular datasets, including both homophilic and heterophilic graphs, with different graph sizes on two typical tasks: node classification and link prediction. Specifically, 1) SkipNode has strong generalizability of being applied to various GCN-based models on different datasets and tasks; and 2) SkipNode outperforms recent state-of-the-art anti-over-smoothing plugand-play modules, i.e., DropEdge and DropNode, in different settings. Code will be made publicly available on GitHub. CCS CONCEPTS • Computing methodologies → Neural networks; Artificial intelligence; Neural networks; • Mathematics of computing → Graph algorithms. This work was done when Weigang was a research intern at JD Explore Academy. Ziyu Guan is the corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Woodstock ’18, June 03–05, 2018, Woodstock, NY © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00 https://doi.org/10.1145/1122445.1122456

Structure-Aware DropEdge Toward Deep Graph Convolutional Networks

Structure-Aware DropEdge Towards Deep Graph Convolutional Networks

Tackling Over-Smoothing for General Graph Convolutional Networks

DII-GCN: Dropedge Based Deep Graph Convolutional Networks

Edge Convolutional Networks: Decomposing Graph Convolutional Networks for Stochastic Training with Independent Edges

SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networks

ADEdgeDrop: Adversarial Edge Dropping for Robust Graph Neural Networks

Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping

Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks

On Dropping Clusters to Regularize Graph Convolutional Neural Networks

Adaptive Sampling Towards Fast Graph Representation Learning

Graph Classification via Discriminative Edge Feature Learning

SkipNode: on Alleviating Over-smoothing for Deep Graph Convolutional Networks.

Going Deep: Graph Convolutional Ladder-Shape Networks

Edge Enhancement Oriented Graph Convolutional Networks for Point Cloud Segmentation

Exploiting Edge Features in Graph Neural Networks

DeeperGCN: All You Need to Train Deeper GCNs

Exploiting Edge Features for Graph Neural Networks

The Transferability of Downsamped Sparse Graph Convolutional Networks

DeepHGCN: Toward Deeper Hyperbolic Graph Convolutional Networks

DeGNN: Improving Graph Neural Networks with Graph Decomposition