Abstract:Graph neural networks (GNNs) are composed of layers consisting of graph convolutions and pointwise nonlinearities. Due to their invariance and stability properties, GNNs are provably successful at learning representations from data supported on moderate-scale graphs. However, they are difficult to learn on large-scale graphs. In this paper, we study the problem of training GNNs on graphs of moderate size and transferring them to large-scale graphs. We use graph limits called graphons to define limit objects for graph filters and GNNs -- graphon filters and graphon neural networks (WNNs) -- which we interpret as generative models for graph filters and GNNs. We then show that graphon filters and WNNs can be approximated by graph filters and GNNs sampled from them on weighted and stochastic graphs. Because the error of these approximations can be upper bounded, by a triangle inequality argument we can further bound the error of transferring a graph filter or a GNN across graphs. Our results show that (i) the transference error decreases with the graph size, and (ii) that graph filters have a transferability-discriminability tradeoff that in GNNs is alleviated by the scattering behavior of the nonlinearity. These findings are demonstrated empirically in a movie recommendation problem and in a decentralized control task.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the training difficulties of graph neural networks (GNNs) on large - scale graphs. Specifically, although GNNs have successfully learned data representations on medium - scale graphs due to their invariance and stability, it is very difficult to train them on large - scale graphs. This is mainly because graph convolution requires complete knowledge of the graph structure, and the structure of large - scale graphs is often difficult to obtain. In addition, graph convolution requires the calculation of large - scale matrix - vector multiplications with a complexity of \(O(n^2)\), which is very costly for non - sparse graphs. To solve this problem, the paper studies the feasibility of training GNNs on medium - scale graphs and transferring them to large - scale graphs. The author uses graphons to define the limit objects of graph filters and GNNs - graphon filters and graphon neural networks (WNNs), and interprets these limit objects as generative models of graph filters and GNNs. In this way, the author proves that graphon filters and WNNs can be approximated by graph filters and GNNs in weighted and random graphs sampled from them, and the error of this approximation can be upper - bounded. Further, through the triangle inequality argument, the author is also able to upper - bound the error of transferring graph filters or GNNs across graphs. The main contributions of the paper are: 1. **Transfer error decreases with graph scale**: The transfer error decreases as the scales of the training graph and the target graph increase. 2. **Trade - off between transferability and spectral resolution ability of graph convolution**: There is a trade - off between the transferability and spectral resolution ability of graph convolution, and in GNNs, this trade - off is alleviated due to the nonlinear scattering behavior. These theoretical results have been empirically verified in the movie recommendation problem and the decentralized control task.

Transferability Properties of Graph Neural Networks

Transferability of Graph Neural Networks using Graphon and Sampling Theories

Graph Neural Networks: Architectures, Stability, and Transferability

Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions

Graph neural networks and non-commuting operators

Training Graph Neural Networks on Growing Stochastic Graphs

Transferability of Spectral Graph Convolutional Neural Networks

Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph Neural Networks

Geometric Graph Filters and Neural Networks: Limit Properties and Discriminability Trade-offs

Discovering the Representation Bottleneck of Graph Neural Networks

An Experimental Study of the Transferability of Spectral Graph Networks

A Manifold Perspective on the Statistical Generalization of Graph Neural Networks

Graph Sampling for Scalable and Expressive Graph Neural Networks on Homophilic Graphs

Graphon Pooling in Graph Neural Networks

Learning to Approximate Adaptive Kernel Convolution on Graphs

Robust Graph Neural Network based on Graph Denoising

Transfer Entropy in Graph Convolutional Neural Networks

Invariance-Preserving Localized Activation Functions for Graph Neural Networks

Graph Policy Network for Transferable Active Learning on Graphs

Are Powerful Graph Neural Nets Necessary? A Dissection on Graph Classification

Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals