Abstract:Over-smoothing is a challenging problem, which degrades the performance of deep graph convolutional networks (GCNs). However, existing studies for alleviating the over-smoothing problem lack either generality or effectiveness. In this paper, we analyze the underlying issues behind the over-smoothing problem, i.e., featurediversity degeneration, gradient vanishing, andmodelweights overdecaying. Inspired by this, we propose a simple yet effective plugand-play module, SkipNode, to alleviate over-smoothing. Specifically, for each middle layer of a GCN model, SkipNode randomly (or based on node degree) selects nodes to skip the convolutional operation by directly feeding their input features to the nonlinear function. Analytically, 1) skipping the convolutional operation prevents the features from losing diversity; and 2) the "skipped" nodes enable gradients to be directly passed back, thus mitigating the gradient vanishing and model weights over-decaying issues. To demonstrate the superiority of SkipNode, we conduct extensive experiments on nine popular datasets, including both homophilic and heterophilic graphs, with different graph sizes on two typical tasks: node classification and link prediction. Specifically, 1) SkipNode has strong generalizability of being applied to various GCN-based models on different datasets and tasks; and 2) SkipNode outperforms recent state-of-the-art anti-over-smoothing plugand-play modules, i.e., DropEdge and DropNode, in different settings. Code will be made publicly available on GitHub. CCS CONCEPTS • Computing methodologies → Neural networks; Artificial intelligence; Neural networks; • Mathematics of computing → Graph algorithms. This work was done when Weigang was a research intern at JD Explore Academy. Ziyu Guan is the corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Woodstock ’18, June 03–05, 2018, Woodstock, NY © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00 https://doi.org/10.1145/1122445.1122456

Handling Over-Smoothing and Over-Squashing in Graph Convolution With Maximization Operation

Multi-scale Graph Convolutional Networks with Self-Attention

SGCN: A Scalable Graph Convolutional Network with Graph-Shaped Kernels and Multi-Channels

ScaleGCN: Efficient and Effective Graph Convolution Via Channel-Wise Scale Transformation.

Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks

MG-GCN: Fast and Effective Learning with Mix-grained Aggregators for Training Large Graph Convolutional Networks

Tackling Over-Smoothing for General Graph Convolutional Networks

Generalizing Graph Convolutional Networks Via Heat Kernel

SkipNode: on Alleviating Over-smoothing for Deep Graph Convolutional Networks.

A novel complex-high-order graph convolutional network paradigm: ChyGCN

Simple and Deep Graph Convolutional Networks

Accel-GCN: High-Performance GPU Accelerator Design for Graph Convolution Networks

Permutohedral-GCN: Graph Convolutional Networks with Global Attention

Graph Convolutional Networks with Multi-Level Coarsening for Graph Classification

Cross-GCN: Enhancing Graph Convolutional Network with K-Order Feature Interactions

Interpretable and Efficient Heterogeneous Graph Convolutional Network

Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective

DyGCN: Efficient Dynamic Graph Embedding With Graph Convolutional Network

Tackling Over-Smoothing in Graph Convolutional Networks With EM-Based Joint Topology Optimization and Node Classification

TSC: A Simple Two-Sided Constraint against Over-Smoothing

Universal Graph Convolutional Networks