Abstract:Graph neural networks (GNNs) have gained significant attention for their impressive results on different graph-based tasks. The essential mechanism of GNNs is the message-passing framework, whereby node representations are aggregated from local neighbourhoods. Recently, Transformer-based GNNs have been introduced to learn the long-range dependencies, enhancing performance. However, their quadratic computational complexity, due to the attention computation, has constrained their applicability on large-scale graphs. To address this issue, we propose MGIGNN ( M emorized G lobal I nformation G raph N eural N etwork), an innovative approach that leverages memorized global information to enhance existing GNNs in both transductive and inductive scenarios. Specifically, MGIGNN captures long-range dependencies by identifying and incorporating global similar nodes, which are defined as nodes exhibiting similar features, structural patterns and label information within a graph. To alleviate the computational overhead associated with computing embeddings for all nodes, we introduce an external memory module to facilitate the retrieval of embeddings and optimize performance on large graphs. To enhance the memory-efficiency, MGIGNN selectively retrieves global similar nodes from a small set of candidate nodes. These candidate nodes are selected from the training nodes based on a sparse node selection distribution with a Dirichlet prior. This selecting approach not only reduces the memory size required but also ensures efficient utilization of computational resources. Through comprehensive experiments conducted on ten widely-used and real-world datasets, including seven homogeneous datasets and three heterogeneous datasets, we demonstrate that our MGIGNN can generally improve the performance of existing GNNs on node classification tasks under both inductive and transductive settings.

Message gain aggregation architecture: a scalable graph neural network for combining large-scale neighborhoods

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via Nonlinear Neighborhood Aggregators

GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing

From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness

All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks

Hierarchical Message-Passing Graph Neural Networks

Enhance Information Propagation for Graph Neural Network by Heterogeneous Aggregations

Multigraph Message Passing with Bi-Directional Multi-Edge Aggregations

GAIN: Graph Attention & Interaction Network for Inductive Semi-Supervised Learning over Large-scale Graphs

Towards Efficient Large-Scale Graph Neural Network Computing.

Graph convolutional network with tree-guided anisotropic message passing

Architectural Implications of GNN Aggregation Programming Abstractions

Revisiting graph neural networks from hybrid regularized graph signal reconstruction

Enhancing Graph Neural Networks via Memorized Global Information

A Unified View on Graph Neural Networks as Graph Signal Denoising

GNN-VPA: A Variance-Preserving Aggregation Strategy for Graph Neural Networks

Large-scale Online Multi-View Graph Neural Network and Applications

Breaking the Expressive Bottlenecks of Graph Neural Networks

Neighbour-level Message Interaction Encoding for Improved Representation Learning on Graphs

Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis