Abstract:Graph neural networks (GNNs) have achieved remarkable success in a variety of machine learning tasks over graph data. Existing GNNs usually rely on message passing, i.e., computing node representations by gathering information from the neighborhood, to build their underlying computational graphs. They are known fairly limited in expressive power, and often fail to capture global characteristics of graphs. To overcome the issue, a popular solution is to use Laplacian eigenvectors as additional node features, as they contain global positional information of nodes, and can serve as extra node identifiers aiding GNNs to separate structurally similar nodes. For such an approach, properly handling the orthogonal group symmetry among eigenvectors with equal eigenvalue is crucial for its stability and generalizability. However, using a naive orthogonal group invariant encoder for each separate eigenspace may not keep the full expressivity in the Laplacian eigenvectors. Moreover, computing such invariants inevitably entails a hard split of Laplacian eigenvalues according to their numerical identity, which suffers from great instability when the graph structure is perturbed. In this paper, we propose a novel method exploiting Laplacian eigenvectors to generate stable and globally expressive graph representations. The main difference from previous works is that (i) our method utilizes learnable orthogonal group invariant representations for each Laplacian eigenspace, based upon powerful orthogonal group equivariant neural network layers already well studied in the literature, and that (ii) our method deals with numerically close eigenvalues in a smooth fashion, ensuring its better robustness against perturbations. Experiments on various graph learning benchmarks witness the competitive performance of our method, especially its great potential to learn global properties of graphs.

Graph Neural Networks for Learning Equivariant Representations of Neural Networks

Graph Metanetworks for Processing Diverse Neural Architectures

Graph Automorphism Group Equivariant Neural Networks

Equivariant Matrix Function Neural Networks

Scale Equivariant Graph Metanetworks

Permutation-Invariant Variational Autoencoder for Graph-Level Representation Learning

Graph Neural Networks Inspired by Classical Iterative Algorithms

Equivariant Neural Operator Learning with Graphon Convolution

Improving Equivariant Graph Neural Networks on Large Geometric Graphs Via Virtual Nodes Learning

Lie Group Decompositions for Equivariant Neural Networks

Towards Stable, Globally Expressive Graph Representations with Laplacian Eigenvectors

Graph Transformer: Learning Better Representations for Graph Neural Networks.

GCNH: A Simple Method For Representation Learning On Heterophilous Graphs

Flexible infinite-width graph convolutional networks and the importance of representation learning

Equivariant Polynomials for Graph Neural Networks

Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks?

SpeqNets: Sparsity-aware Permutation-equivariant Graph Networks

Feature Expansion for Graph Neural Networks

Graph Neural Networks for Small Graph and Giant Network Representation Learning: An Overview

Relaxing Continuous Constraints of Equivariant Graph Neural Networks for Physical Dynamics Learning

Equivariant Machine Learning on Graphs with Nonlinear Spectral Filters