Abstract:Graph neural networks (GNNs) have gained significant attention for their impressive results on different graph-based tasks. The essential mechanism of GNNs is the message-passing framework, whereby node representations are aggregated from local neighbourhoods. Recently, Transformer-based GNNs have been introduced to learn the long-range dependencies, enhancing performance. However, their quadratic computational complexity, due to the attention computation, has constrained their applicability on large-scale graphs. To address this issue, we propose MGIGNN ( M emorized G lobal I nformation G raph N eural N etwork), an innovative approach that leverages memorized global information to enhance existing GNNs in both transductive and inductive scenarios. Specifically, MGIGNN captures long-range dependencies by identifying and incorporating global similar nodes, which are defined as nodes exhibiting similar features, structural patterns and label information within a graph. To alleviate the computational overhead associated with computing embeddings for all nodes, we introduce an external memory module to facilitate the retrieval of embeddings and optimize performance on large graphs. To enhance the memory-efficiency, MGIGNN selectively retrieves global similar nodes from a small set of candidate nodes. These candidate nodes are selected from the training nodes based on a sparse node selection distribution with a Dirichlet prior. This selecting approach not only reduces the memory size required but also ensures efficient utilization of computational resources. Through comprehensive experiments conducted on ten widely-used and real-world datasets, including seven homogeneous datasets and three heterogeneous datasets, we demonstrate that our MGIGNN can generally improve the performance of existing GNNs on node classification tasks under both inductive and transductive settings.

IncreGNN: Incremental Graph Neural Network Learning by Considering Node and Parameter Importance

Streaming Graph Neural Networks Via Continual Learning

Towards Robust Graph Incremental Learning on Evolving Graphs

Streaming Graph Neural Networks with Generative Replay

Graph Relearn Network: Reducing Performance Variance and Improving Prediction Accuracy of Graph Neural Networks

Deep Graph Neural Networks via Flexible Subgraph Aggregation

Instant Graph Neural Networks for Dynamic Graphs

GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems

Inductive Graph Few-shot Class Incremental Learning

FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Training

TinyGNN: Learning Efficient Graph Neural Networks

Incremental Graph Classification by Class Prototype Construction and Augmentation

Enhancing Graph Neural Networks via Memorized Global Information

Adaptive Graph Neural Network with Incremental Learning Mechanism for Knowledge Graph Reasoning

Incremental Graph Convolutional Network for Collaborative Filtering

InkStream: Real-time GNN Inference on Streaming Graphs via Incremental Update

AKE-GNN: Effective Graph Learning with Adaptive Knowledge Exchange

Training Graph Neural Networks on Growing Stochastic Graphs

Universal Inceptive GNNs by Eliminating the Smoothness-generalization Dilemma

Ada-GNN: Adapting to Local Patterns for Improving Graph Neural Networks

Inflation Improves Graph Neural Networks