Traffic Prediction Based on a Novel Deep Learning Model: TPBERT

Min Gao,Yingmei Wei
DOI: https://doi.org/10.1109/ICAIBD57115.2023.10206156
2023-01-01
Abstract:The intelligent transportation system (ITS) was born to cope with increasingly complex traffic conditions. Traffic prediction is an essential part of ITS, which can help to prevent traffic congestion and reduce traffic accidents. Traffic prediction has two major challenges: temporal dependencies and spatial dependencies. Traditional statistical methods and machine learning methods focus on temporal correlations and ignore spatial correlations. Meanwhile, traditional methods still suffer from long-term dependencies in temporal modeling and tend to forget traffic states from a long time ago. Since deep learning methods have the potential to deal with two challenges at the same time, we propose a novel traffic prediction model based on deep learning named TPBERT in this paper. TPBERT is inspired by the milestone model in NLP called Bidirectional Encoder Representation from Transformer (BERT), which solves the problem of long-term dependencies. We propose two input representations to represent the historical and future traffic information respectively. Both of them include temporal embedding, spatial embedding, and positional embedding. Besides, we propose a separated time-space attention mechanism to address two primary challenges of traffic prediction. TPBERT is evaluated on two real-world traffic datasets: METR-LA and PeMS-BAY. We choose several traffic prediction models as baselines and compare them with TPBERT in terms of prediction accuracy. The experimental results show that TPBERT outperforms the baselines in long-term prediction. Further, we conduct ablation studies to analyze the role of each module in TPBERT. The results show that each embedding in the input representations helps to improve the prediction accuracy, and separated time-space attention is more effective than individual space attention or time attention.
What problem does this paper attempt to address?