Abstract:Graph Neural Networks (GNNs) have been successfully applied to many real-world static graphs. However, the success of static graphs has not fully translated to dynamic graphs due to the limitations in model design, evaluation settings, and training strategies. Concretely, existing dynamic GNNs do not incorporate state-of-the-art designs from static GNNs, which limits their performance. Current evaluation settings for dynamic GNNs do not fully reflect the evolving nature of dynamic graphs. Finally, commonly used training methods for dynamic GNNs are not scalable. Here we propose ROLAND, an effective graph representation learning framework for real-world dynamic graphs. At its core, the ROLAND framework can help researchers easily repurpose any static GNN to dynamic graphs. Our insight is to view the node embeddings at different GNN layers as hierarchical node states and then recurrently update them over time. We then introduce a live-update evaluation setting for dynamic graphs that mimics real-world use cases, where GNNs are making predictions and being updated on a rolling basis. Finally, we propose a scalable and efficient training approach for dynamic GNNs via incremental training and meta-learning. We conduct experiments over eight different dynamic graph datasets on future link prediction tasks. Models built using the ROLAND framework achieve on average 62.7% relative mean reciprocal rank (MRR) improvement over state-of-the-art baselines under the standard evaluation settings on three datasets. We find state-of-the-art baselines experience out-of-memory errors for larger datasets, while ROLAND can easily scale to dynamic graphs with 56 million edges. After re-implementing these baselines using the ROLAND training strategy, ROLAND models still achieve on average 15.5% relative MRR improvement over the baselines.

A General Benchmark Framework is Dynamic Graph Neural Network Need

A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges

Deep learning for dynamic graphs: models and benchmarks

A general deep learning framework for network reconstruction and dynamics learning

Dynamic Graph Representation Learning With Neural Networks: A Survey

Dynamic Spiking Framework for Graph Neural Networks

UGSL: A Unified Framework for Benchmarking Graph Structure Learning

A Survey on Dynamic Graph Neural Networks Modeling

Temporal receptive field in dynamic graph learning: A comprehensive analysis

A survey of dynamic graph neural networks

Scaling Up Dynamic Graph Representation Learning Via Spiking Neural Networks

ROLAND: Graph Learning Framework for Dynamic Graphs

Revisiting the Necessity of Graph Learning and Common Graph Benchmarks

Efficient and Effective Implicit Dynamic Graph Neural Network

DynaGraph: dynamic graph neural networks at scale

Instant Graph Neural Networks for Dynamic Graphs

Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey

Temporal Graph Networks for Deep Learning on Dynamic Graphs

A recurrent graph neural network for inductive representation learning on dynamic graphs

Dynamic Neural Networks: A Survey

Dynamic network link prediction with node representation learning from graph convolutional networks