Abstract:Graph Neural Networks (GNNs) have received considerable attention on graph-structured data learning for a wide variety of tasks. The well-designed propagation mechanism, which has been demonstrated effective, is the most fundamental part of GNNs. Although most of the GNNs basically follow a message passing manner, little effort has been made to discover and analyze their essential relations. In this paper, we establish a surprising connection between different propagation mechanisms with an optimization problem. We show that despite the proliferation of various GNNs, in fact, their proposed propagation mechanisms are the optimal solutions of a generalized optimization framework with a flexible feature fitting function and a generalized graph regularization term. Actually, the optimization framework can not only help understand the propagation mechanisms of GNNs, but also open up opportunities for flexibly designing new GNNs. Through analyzing the general solutions of the optimization framework, we provide a more convenient way for deriving corresponding propagation results of GNNs. We further discover that existing works usually utilize naïve graph convolutional kernels for feature fitting function, or just utilize one-hop structural information (original topology graph) for graph regularization term. Correspondingly, we develop two novel objective functions considering adjustable graph kernels showing low-pass or high-pass filtering capabilities and one novel objective function considering high-order structural information during propagation respectively. Extensive experiments on benchmark datasets clearly show that the newly proposed GNNs not only outperform the state-of-the-art methods but also have good ability to alleviate over-smoothing, and further verify the feasibility for designing GNNs with the generalized unified optimization framework.

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Unifying Graph Neural Networks with a Generalized Optimization Framework

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

A Unified View on Graph Neural Networks as Graph Signal Denoising

Interpreting and Unifying Graph Neural Networks with an Optimization Framework

Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks

Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View

Node Dependent Local Smoothing for Scalable Graph Learning

Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs

Improving Your Graph Neural Networks: A High-Frequency Booster

A Survey on Oversmoothing in Graph Neural Networks

Graph Neural Networks Inspired by Classical Iterative Algorithms

Implicit Graph Neural Diffusion Networks: Convergence, Generalization, and Over-Smoothing

SmoothGNN: Smoothing-aware GNN for Unsupervised Node Anomaly Detection

Revisiting graph neural networks from hybrid regularized graph signal reconstruction

Measuring and Improving the Use of Graph Information in Graph Neural Networks

Graph Neural Networks Do Not Always Oversmooth

Graph Dilated Network with Rejection Mechanism

Learning to Control the Smoothness of Graph Convolutional Network Features

Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth