Abstract:Graph Neural Networks (GNNs) have received considerable attention on graph-structured data learning for a wide variety of tasks. The well-designed propagation mechanism, which has been demonstrated effective, is the most fundamental part of GNNs. Although most of the GNNs basically follow a message passing manner, little effort has been made to discover and analyze their essential relations. In this paper, we establish a surprising connection between different propagation mechanisms with an optimization problem. We show that despite the proliferation of various GNNs, in fact, their proposed propagation mechanisms are the optimal solutions of a generalized optimization framework with a flexible feature fitting function and a generalized graph regularization term. Actually, the optimization framework can not only help understand the propagation mechanisms of GNNs, but also open up opportunities for flexibly designing new GNNs. Through analyzing the general solutions of the optimization framework, we provide a more convenient way for deriving corresponding propagation results of GNNs. We further discover that existing works usually utilize naïve graph convolutional kernels for feature fitting function, or just utilize one-hop structural information (original topology graph) for graph regularization term. Correspondingly, we develop two novel objective functions considering adjustable graph kernels showing low-pass or high-pass filtering capabilities and one novel objective function considering high-order structural information during propagation respectively. Extensive experiments on benchmark datasets clearly show that the newly proposed GNNs not only outperform the state-of-the-art methods but also have good ability to alleviate over-smoothing, and further verify the feasibility for designing GNNs with the generalized unified optimization framework.

A Unified Framework for Convolution-Based Graph Neural Networks

A Unified Framework for Convolution-based Graph Neural Networks

Graph Learning in Low Dimensional Space for Graph Convolutional Networks

Unified GCNs: Towards Connecting GCNs with CNNs

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Beyond Graph Convolutional Network: an Interpretable Regularizer-centered Optimization Framework

Interpreting and Unifying Graph Neural Networks with an Optimization Framework

Simple and Deep Graph Convolutional Networks

Universal Graph Convolutional Networks

Multi-scale Graph Convolutional Networks with Self-Attention

Unifying and Improving Graph Convolutional Neural Networks with Wavelet Denoising Filters

Generalizing Graph Convolutional Networks Via Heat Kernel

Unifying Graph Neural Networks with a Generalized Optimization Framework

Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning

Bridging the Gap Between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networks

Simplifying Graph Convolutional Networks

Probabilistic Graph Convolutional Network Via Topology-Constrained Latent Space Model.

Hypergraph Convolutional Networks via Equivalency between Hypergraphs and Undirected Graphs

Understanding the Effect of GCN Convolutions in Regression Tasks

A novel complex-high-order graph convolutional network paradigm: ChyGCN

Learning Discrete Adaptive Receptive Fields for Graph Convolutional Networks