Abstract:Temporal graph neural networks (T-GNNs) have emerged as leading approaches for representation learning over dynamic graphs. However, existing solutions typically suffer from exponential time complexity with model depth and excessive GPU memory usage due to acceleration techniques, and cannot handle large dynamic graphs. Furthermore, the core component of T-G NNs, temporal message passing, still predominantly derives from static GNNs. This neglects the distinct characteristics of two types of features, timestamps and edge features, and results in sub-optimal embedding quality. Consequently, existing T-GNNs fail to scale to large dynamic graphs and generalize well in unseen or complex scenarios, limiting their applicability. To bridge the gap, this paper first proposes a simple yet effective temporal message passing paradigm for T-GNNs, called the divided temporal message passing (DT-MP) paradigm, which enables effective feature learning for each feature type. We theoretically demonstrate that the DT-MP paradigm can reduce GPU memory usage compared to existing T-GNNs. Building on this foundation, we propose TimeSGN, a scalable and effective temporal graph neural network, which can handle billion-scale dynamic graphs. Specifically, we design a linear state updater to effectively capture node dynamic evolution and instantiate the DT-MP paradigm using two 1-layer self-attention mechanisms for temporal message passing to generate temporal embeddings. As a result, TimeSGN fundamentally avoids exponential time complexity and significantly reduces GPU memory usage. Extensive experiments demonstrate that TimeSGN achieves an average 10.56% improvement in accuracy, up to 42.48% reduction in training GPU memory, and up to 5 x speedup in per-epoch training time compared to the state-of-the-art baselines, while being one order of magnitude faster than vanilla T-GNNs.

Learning Continuous Dynamic Network Representation with Transformer-Based Temporal Graph Neural Network.

Temporal Graph Transformer for Dynamic Network

Continuous Temporal Graph Networks for Event-Based Graph Data

Temporal Graph Network for continuous-time dynamic event sequence

DTFormer: A Transformer-Based Method for Discrete-Time Dynamic Graph Representation Learning

Dynamic Graph Representation Learning via Edge Temporal States Modeling and Structure-reinforced Transformer

Continuous-Time Dynamic Graph Learning Via Neural Interaction Processes.

Tensor Graph Convolutional Network for Dynamic Graph Representation Learning

Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation

Temporal Graph Networks for Deep Learning on Dynamic Graphs

Spatial-temporal Graph Convolutional Networks with Diversified Transformation for Dynamic Graph Representation Learning

Scalable and Effective Temporal Graph Representation Learning With Hyperbolic Geometry

TimeSGN: Scalable and Effective Temporal Graph Neural Network

Dynamic Graph Representation Based on Temporal and Contextual Contrasting

Dynamic Graph Representation Based on Temporal and Contextual Contrasting.

SiGNN: A Spike-induced Graph Neural Network for Dynamic Graph Representation Learning

Dynamic Representation Learning Via Recurrent Graph Neural Networks

Retrofitting Temporal Graph Neural Networks with Transformer

A Hybrid Continuous-Time Dynamic Graph Representation Learning Model by Exploring Both Temporal and Repetitive Information

DGNN-MN: Dynamic Graph Neural Network via memory regenerate and neighbor propagation

TI-GCN: A Dynamic Network Embedding Method with Time Interval Information