TASER: Temporal Adaptive Sampling for Fast and Accurate Dynamic Graph Representation Learning
Gangda Deng, Hongkuan Zhou, Hanqing Zeng, Yinglong Xia, Christopher Leung, Jianbo Li, Rajgopal Kannan, Viktor Prasanna
2024-02-08
Abstract:Recently, Temporal Graph Neural Networks (TGNNs) have demonstrated
state-of-the-art performance in various high-impact applications, including
fraud detection and content recommendation. Despite the success of TGNNs, they
are prone to the prevalent noise found in real-world dynamic graphs like
time-deprecated links and skewed interaction distribution. The noise causes two
critical issues that significantly compromise the accuracy of TGNNs: (1) models
are supervised by inferior interactions, and (2) noisy input induces high
variance in the aggregated messages. However, current TGNN denoising techniques
do not consider the diverse and dynamic noise pattern of each node. In
addition, they also suffer from the excessive mini-batch generation overheads
caused by traversing more neighbors. We believe the remedy for fast and
accurate TGNNs lies in temporal adaptive sampling. In this work, we propose
TASER, the first adaptive sampling method for TGNNs optimized for accuracy,
efficiency, and scalability. TASER adapts its mini-batch selection based on
training dynamics and temporal neighbor selection based on the contextual,
structural, and temporal properties of past interactions. To alleviate the
bottleneck in mini-batch generation, TASER implements a pure GPU-based temporal
neighbor finder and a dedicated GPU feature cache. We evaluate the performance
of TASER using two state-of-the-art backbone TGNNs. On five popular datasets,
TASER outperforms the corresponding baselines by an average of 2.3% in Mean
Reciprocal Rank (MRR) while achieving an average of 5.1x speedup in training
time.
Artificial Intelligence,Machine Learning