Abstract:The way humans travel and even their daily commute, is gradually expanding beyond the confines of counties and cities. Traffic between counties, cities, and even across the entire state is increasingly becoming a common aspect of daily activities. The demand for traffic flow forecasting covering larger geographical areas and longer time spans is ongoing. However, existing studies lack targeted deep model proposals for large-scale forecasting. To address this gap, we propose Spatiotemporal Fusion Transformer (STFT). Specifically, we propose three modules on top of the Transformer architecture: (i) Seasonality Encoding, based on the multi-periodicity inherent in traffic flow to facilitate the extraction of more predictable time-variant components from complex patterns. (ii) Tubelet Embedding, partitioning the input into Tubelets as input tokens for the Transformer. The Tubelet design not only achieves quadratic reductions in computational and memory usage but also enhances spatiotemporal locality feature modelling. (iii) Token Permutator, leveraging diffusion graph to model the spatiotemporal dynamics as a token permutation process. The graph representation is then projected by a proposed Hadamard Mapper to circumvent the anomaly sensitivities of Graph Neural Networks in large-scale computations. Experimental results on five real-world datasets indicate that STFT can cater to collaborative forecasting at diverse scales (subdivision, county, municipal, state) that not only outperforms state-of-the-art methods but also enjoys a large speedup of up to 4.46×. Lastly, we also find that compared to independent forecasting for each subregion, large-scale collaborative forecasting with STFT offers both better feature utilization and requires less computational cost.

uTransformer: unified spatial-temporal transformer with external factors for traffic flow forecasting

Foreformer: an Enhanced Transformer-Based Framework for Multivariate Time Series Forecasting

Short-Term Speed Forecasting of Large-Scale Urban Road Network Based on Transformer

Transformer network with decoupled spatial–temporal embedding for traffic flow forecasting

A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

A Multi-Channel Spatial-Temporal Transformer Model for Traffic Flow Forecasting

MVSTT: A Multiview Spatial-Temporal Transformer Network for Traffic-Flow Forecasting

Spatiotemporal Fusion Transformer for large-scale traffic forecasting

STA-former: encoding traffic flows with spatio-temporal associations in transformer networks for prediction

Bidirectional Spatial-Temporal Adaptive Transformer for Urban Traffic Flow Forecasting

Dual-Transformer: A General Model for Traffic Accident Prediction

STTF: An Efficient Transformer Model for Traffic Congestion Prediction

Trafformer: Unify Time and Space in Traffic Prediction.

Adaptive Spatiotemporal Transformer Graph Network for Traffic Flow Forecasting by IoT Loop Detectors

Learning dynamic and hierarchical traffic spatiotemporal features with Transformer

MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization

Traffic flow prediction based on transformer

Spatial linear transformer and temporal convolution network for traffic flow prediction

ASSTFormer: Adaptive Sparse Spatial-Temporal Transformer for Effective Traffic Forecasting

Boosting Urban Traffic Speed Prediction via Integrating Implicit Spatial Correlations

Rethinking Spatio-Temporal Transformer for Traffic Prediction:Multi-level Multi-view Augmented Learning Framework