Abstract:As many real-world applications are streaming and attached with time instances, a few works have been proposed to learn streaming graph neural networks (GNNs). Unfortunately, current streaming GNNs are observed to have a large training overhead and suffer from bad parallel scalability on multiple GPUs. These drawbacks pose severe challenges to online learning of streaming GNNs and their application to real-time scenarios. To improve training efficiency, one promising solution is to use sampling, a technique widely used in static GNNs. However, to the best of our knowledge, sampling has not been investigated in learning streaming GNNs. Based on these observations, in this paper, we propose T-GCN, the first sampling-based streaming GNN system, which targets temporal-aware streaming graphs and takes advantage of a hybrid CPU-GPU co-processing architecture to achieve high throughput and low latency. T-GCN proposes an efficient sampling method, namely Segment Its Search , to offer high sampling speed with respect to three typical types of general graph sampling methods (i.e., node-wise, layer-wise, and subgraph sampling). We propose a locality-aware data partitioning method to reduce CPU-GPU communication latency and data transfer overhead, and an NVLink-specific task schedule to fully exploit NVLink's fast speed and improve GPU-GPU communication efficiency. Besides, we further pipeline the computation and the communication by introducing an efficient memory management mechanism, to improve scalability while hiding data communication. Overall, with respect to end-to-end performance, for single-GPU training, T-GCN achieves up to 7.9× speedup than state-of-the-art works. In terms of scalability, T-GCN runs 5.2× faster on average with 8 GPUs than one GPU. Additionally, in terms of sampling, T-GCN also yields a maximum of 38.8× speedup with our Segment Its Search sampling method.

Streaming Graph Neural Networks with Generative Replay

Streaming Graph Neural Networks Via Continual Learning

InkStream: Real-time GNN Inference on Streaming Graphs via Incremental Update

FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Training

On the Limitation and Experience Replay for GNNs in Continual Learning

D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks

Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay

T-GCN: A Sampling Based Streaming Graph Neural Network System with Hybrid Architecture.

NeutronStream: A Dynamic GNN Training Framework with Sliding Window for Graph Streams

Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction

Haste Makes Waste: A Simple Approach for Scaling Graph Neural Networks

Graph Continual Learning with Debiased Lossless Memory Replay

Generative Feature Replay with Orthogonal Weight Modification for Continual Learning

GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs

STAG: Enabling Low Latency and Low Staleness of GNN-based Services with Dynamic Graphs

Neural Memory Streaming Recommender Networks with Adversarial Training.

Graph Neural Networks Inspired by Classical Iterative Algorithms

Scalable Spatiotemporal Graph Neural Networks

Class-Incremental Learning Using Generative Experience Replay Based on Time-aware Regularization

Sparsified Subgraph Memory for Continual Graph Representation Learning