Abstract:Graph neural networks (GNNs) have gained significant attention for their impressive results on different graph-based tasks. The essential mechanism of GNNs is the message-passing framework, whereby node representations are aggregated from local neighbourhoods. Recently, Transformer-based GNNs have been introduced to learn the long-range dependencies, enhancing performance. However, their quadratic computational complexity, due to the attention computation, has constrained their applicability on large-scale graphs. To address this issue, we propose MGIGNN ( M emorized G lobal I nformation G raph N eural N etwork), an innovative approach that leverages memorized global information to enhance existing GNNs in both transductive and inductive scenarios. Specifically, MGIGNN captures long-range dependencies by identifying and incorporating global similar nodes, which are defined as nodes exhibiting similar features, structural patterns and label information within a graph. To alleviate the computational overhead associated with computing embeddings for all nodes, we introduce an external memory module to facilitate the retrieval of embeddings and optimize performance on large graphs. To enhance the memory-efficiency, MGIGNN selectively retrieves global similar nodes from a small set of candidate nodes. These candidate nodes are selected from the training nodes based on a sparse node selection distribution with a Dirichlet prior. This selecting approach not only reduces the memory size required but also ensures efficient utilization of computational resources. Through comprehensive experiments conducted on ten widely-used and real-world datasets, including seven homogeneous datasets and three heterogeneous datasets, we demonstrate that our MGIGNN can generally improve the performance of existing GNNs on node classification tasks under both inductive and transductive settings.

Optimizing Memory Efficiency of Graph NeuralNetworks on Edge Computing Platforms

Brief Industry Paper: optimizing Memory Efficiency of Graph Neural Networks on Edge Computing Platforms

Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective

Unlocking the Non-deterministic Computing Power with Memory-Elastic Multi-Exit Neural Networks

G-NMP: Accelerating Graph Neural Networks with DIMM-based Near-Memory Processing

GNNear: Accelerating Full-Batch Training of Graph Neural Networks with Near-Memory Processing

GNN at the Edge: Cost-Efficient Graph Neural Network Processing over Distributed Edge Servers

FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Training

HGNAS: Hardware-Aware Graph Neural Architecture Search for Edge Devices

Hardware-Aware Graph Neural Network Automated Design for Edge Computing Platforms

Towards Efficient Point Cloud Graph Neural Networks Through Architectural Simplification

DeGNN: Improving Graph Neural Networks with Graph Decomposition

Sparse Decomposition of Graph Neural Networks

MEGA: A Memory-Efficient GNN Accelerator Exploiting Degree-Aware Mixed-Precision Quantization

MAGIS: Memory Optimization Via Coordinated Graph Transformation and Scheduling for DNN

GATe: Streamlining Memory Access and Communication to Accelerate Graph Attention Network With Near-Memory Processing

IMA-GNN: In-Memory Acceleration of Centralized and Decentralized Graph Neural Networks at the Edge

Enhancing Graph Neural Networks via Memorized Global Information

SpanGNN: Towards Memory-Efficient Graph Neural Networks via Spanning Subgraph Training

Alternately Optimized Graph Neural Networks