Abstract:Graph Neural Networks (GNNs) have shown great promise in modeling relationships between nodes in a graph, but capturing higher-order relationships remains a challenge for large-scale networks. Previous studies have primarily attempted to utilize the information from higher-order neighbors in the graph, involving the incorporation of powers of the shift operator, such as the graph Laplacian or adjacency matrix. This approach comes with a trade-off in terms of increased computational and memory demands. Relying on graph spectral theory, we make a fundamental observation: the regular and the Hadamard power of the Laplacian matrix behave similarly in the spectrum. This observation has significant implications for capturing higher-order information in GNNs for various tasks such as node classification and semi-supervised learning. Consequently, we propose a novel graph convolutional operator based on the sparse Sobolev norm of graph signals. Our approach, known as Sparse Sobolev GNN (S2-GNN), employs Hadamard products between matrices to maintain the sparsity level in graph representations. S2-GNN utilizes a cascade of filters with increasing Hadamard powers to generate a diverse set of functions. We theoretically analyze the stability of S2-GNN to show the robustness of the model against possible graph perturbations. We also conduct a comprehensive evaluation of S2-GNN across various graph mining, semi-supervised node classification, and computer vision tasks. In particular use cases, our algorithm demonstrates competitive performance compared to state-of-the-art GNNs in terms of performance and running time.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to efficiently capture high - order relationships between nodes in large - scale networks. Specifically, although existing graph neural networks (GNNs) have shown great potential in modeling relationships between nodes in graphs, capturing higher - order relationships remains a challenge when dealing with large - scale networks. Previous research mainly attempted to solve this problem by using high - order neighbor information in graphs, which involves using power operations of the graph Laplacian matrix or the adjacency matrix. However, this method will increase computational and memory requirements, leading to reduced efficiency. To solve the above problems, this paper proposes a new graph convolution operator based on the sparse Sobolev norm of graph signals. This method is called the Sparse Sobolev Graph Neural Network (S2 - GNN). S2 - GNN maintains the sparsity level of graph representations by using the Hadamard product between matrices and generates diverse function sets using filter cascades. In addition, the paper also theoretically analyzes the stability of S2 - GNN, demonstrating the robustness of the model to possible graph perturbations. Experimental results show that S2 - GNN performs excellently in multiple graph mining tasks, semi - supervised node classification tasks, and computer vision tasks, and is competitive in performance and running time compared with existing state - of - the - art GNN methods. In summary, the main contributions of this paper include: 1. Discovering that the spectra of the high - order Laplacian matrices of weighted graphs are similar to those of their sparse counterparts. This discovery makes it possible to design GNN architectures with superior or comparable performance while maintaining efficiency. 2. Proposing S2 - GNN, a new GNN architecture that utilizes high - order sparse filter cascades, effectively using high - order sparse operations without overly increasing complexity. 3. Conducting a strict theoretical analysis of the stability of S2 - GNN, revealing its robustness to possible graph perturbations. 4. Conducting extensive experimental evaluations on multiple publicly available benchmark datasets and comparing with nine state - of - the - art GNN methods. The results show that S2 - GNN outperforms or is comparable to existing methods in performance.

Higher-Order GNNs Meet Efficiency: Sparse Sobolev Graph Neural Networks

Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization

Global Minima, Recoverability Thresholds, and Higher-Order Structure in GNNS

Spatio-Spectral Graph Neural Networks

Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks

Graph Neural Network Meets Sparse Representation: Graph Sparse Neural Networks Via Exclusive Group Lasso

Improving the interpretability of GNN predictions through conformal-based graph sparsification

A2GCN: Graph Convolutional Networks with Adaptive Frequency and Arbitrary Order

SGCN: A Scalable Graph Convolutional Network with Graph-Shaped Kernels and Multi-Channels

From Local to Global: Spectral-Inspired Graph Neural Networks

Alleviating Over-Smoothing Via Graph Sparsification Based on Vertex Feature Similarity

Scalable Graph Neural Networks for Heterogeneous Graphs

Graph Neural Networks Inspired by Classical Iterative Algorithms

Graph Neural Networks Need Cluster-Normalize-Activate Modules

Graph Neural Networks with High-Order Polynomial Spectral Filters.

Dynamic Spiking Framework for Graph Neural Networks

How Powerful are Graph Neural Networks?

From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness

Spiking Graph Convolutional Networks

Universal Inceptive GNNs by Eliminating the Smoothness-generalization Dilemma

GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning over Large-scale Graphs