Abstract:Temporal Graph Neural Networks (TGNNs) have emerged as powerful tools for modeling dynamic interactions across various domains. The design space of TGNNs is notably complex, given the unique challenges in runtime efficiency and scalability raised by the evolving nature of temporal graphs. We contend that many of the existing works on TGNN modeling inadequately explore the design space, leading to suboptimal designs. Viewing TGNN models through a performance-focused lens often obstructs a deeper understanding of the advantages and disadvantages of each technique. Specifically, benchmarking efforts inherently evaluate models in their original designs and implementations, resulting in unclear accuracy comparisons and misleading runtime. To address these shortcomings, we propose a practical comparative evaluation framework that performs a design space search across well-known TGNN modules based on a unified, optimized code implementation. Using our framework, we make the first efforts towards addressing three critical questions in TGNN design, spending over 10,000 GPU hours: (1) investigating the efficiency of TGNN module designs, (2) analyzing how the effectiveness of these modules correlates with dataset patterns, and (3) exploring the interplay between multiple modules. Key outcomes of this directed investigative approach include demonstrating that the most recent neighbor sampling and attention aggregator outperform uniform neighbor sampling and MLP-Mixer aggregator; Assessing static node memory as an effective node memory alternative, and showing that the choice between static or dynamic node memory should be based on the repetition patterns in the dataset. Our in-depth analysis of the interplay between TGNN modules and dataset patterns should provide a deeper insight into TGNN performance along with potential research directions for designing more general and effective TGNNs.

Temporal Graph Benchmark for Machine Learning on Temporal Graphs

TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs

An Empirical Evaluation of Temporal Graph Benchmark

Datasets for Paper "benchtemp: A General Benchmark for Evaluating Temporal Graph Neural Networks"

Temporal Graph Analysis with TGX

Introduction to a Temporal Graph Benchmark

DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs

Open Graph Benchmark: Datasets for Machine Learning on Graphs

Analysis of different temporal graph neural network configurations on dynamic graphs

Arxiv4tgc: Large-Scale Datasets for Temporal Graph Clustering.

Temporal Graph Generation Featuring Time-Bound Communities

Towards Ideal Temporal Graph Neural Networks: Evaluations and Conclusions after 10,000 GPU Hours

TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs

Temporal receptive field in dynamic graph learning: A comprehensive analysis

LasTGL: An Industrial Framework for Large-Scale Temporal Graph Learning

Scalable and Efficient Temporal Graph Representation Learning via Forward Recent Sampling

TimeGraphs: Graph-based Temporal Reasoning

ETC: Efficient Training of Temporal Graph Neural Networks over Large-Scale Dynamic Graphs

Temporal Graph Networks for Deep Learning on Dynamic Graphs

Deep Temporal Graph Clustering

SIMPLE: Efficient Temporal Graph Neural Network Training at Scale with Dynamic Data Placement.