Abstract:With the rapid development of the Intelligent Transportation System (ITS), accurate traffic forecasting has emerged as a critical challenge. The key bottleneck lies in capturing the intricate spatio-temporal traffic patterns. In recent years, numerous neural networks with complicated architectures have been proposed to address this issue. However, the advancements in network architectures have encountered diminishing performance gains. In this study, we present a novel component called spatio-temporal adaptive embedding that can yield outstanding results with vanilla transformers. Our proposed Spatio-Temporal Adaptive Embedding transformer (STAEformer) achieves state-of-the-art performance on five real-world traffic forecasting datasets. Further experiments demonstrate that spatio-temporal adaptive embedding plays a crucial role in traffic forecasting by effectively capturing intrinsic spatio-temporal relations and chronological information in traffic time series.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that, in the context of the rapid development of intelligent transportation systems (ITS), accurate traffic prediction has become a key challenge. Specifically, the main bottleneck in traffic prediction lies in capturing complex spatio - temporal traffic patterns. In recent years, although many neural networks with complex architectures have been proposed to solve this problem, the progress of these architectures has encountered the problem of a gradually slowing performance improvement. To solve these problems, the paper proposes a new component - spatio - temporal adaptive embedding, which can achieve excellent performance on the standard Transformer model. The model proposed in the paper is called Spatio - Temporal Adaptive Embedding Transformer (STAEformer), which has achieved state - of - the - art performance on six real - world traffic prediction datasets. Further experiments show that spatio - temporal adaptive embedding plays a key role in traffic prediction and can effectively capture the inherent spatio - temporal relationships and time - order information in traffic time series. ### Main Contributions 1. **Propose spatio - temporal adaptive embedding**: By introducing a new spatio - temporal adaptive embedding method, this method can significantly improve the performance of traffic prediction on the standard Transformer model. 2. **Simplify the model architecture**: Compared with existing complex models, STAEformer has a more concise architecture, but it can still reach or exceed the performance of existing methods. 3. **Extensive experimental verification**: Extensive experiments have been carried out on six real - world traffic prediction datasets, proving the effectiveness of the proposed method. ### Method Overview - **Input embedding layer**: Obtain feature embedding through a fully - connected layer, and obtain periodicity embedding through day - of - week embedding and timestamps - of - day embedding. In addition, spatio - temporal adaptive embedding is also designed to capture complex spatio - temporal relationships. - **Transformer layer**: Apply standard Transformer layers on the time axis and the space axis respectively to capture the complex relationships in traffic data. - **Regression layer**: Finally, generate prediction results through a regression layer. ### Experimental Results - **Performance comparison**: STAEformer outperforms a variety of existing methods in multiple metrics (such as MAE, RMSE, and MAPE), especially when graph modeling is not used, the performance improvement is particularly significant. - **Ablation study**: By removing different embedding layers and Transformer layers, the influence of each part on the model performance is verified, further proving the importance of spatio - temporal adaptive embedding. ### Conclusion By introducing spatio - temporal adaptive embedding, the paper provides a simple and effective method that can achieve state - of - the - art performance in traffic prediction on the standard Transformer model. This provides a new direction for solving the complex spatio - temporal relationships in traffic prediction.

STAEformer: Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting

Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting

Short-Term Speed Forecasting of Large-Scale Urban Road Network Based on Transformer

ASSTFormer: Adaptive Sparse Spatial-Temporal Transformer for Effective Traffic Forecasting

STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting

STA-former: encoding traffic flows with spatio-temporal associations in transformer networks for prediction

Transformer network with decoupled spatial–temporal embedding for traffic flow forecasting

A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

TST-Trans: A Transformer Network for Urban Traffic Flow Prediction

Spatiotemporal Fusion Transformer for large-scale traffic forecasting

STEFT: Spatio-Temporal Embedding Fusion Transformer for Traffic Prediction

Adaptive Spatiotemporal Transformer Graph Network for Traffic Flow Forecasting by IoT Loop Detectors

Efficient Large-Scale Traffic Forecasting with Transformers: A Spatial Data Management Perspective

Spatio-Temporal AutoEncoder for Traffic Flow Prediction

ProSTformer: Pre-trained Progressive Space-Time Self-attention Model for Traffic Flow Forecasting

Adaptive Graph Spatial-Temporal Transformer Network for Traffic Flow Forecasting

Rethinking Spatio-Temporal Transformer for Traffic Prediction:Multi-level Multi-view Augmented Learning Framework

Bidirectional Spatial-Temporal Adaptive Transformer for Urban Traffic Flow Forecasting

STTF: An Efficient Transformer Model for Traffic Congestion Prediction

A Multi-view Spatial-temporal Adaptive Transformer-GRU Framework for Traffic Flow Prediction

Learning dynamic and hierarchical traffic spatiotemporal features with Transformer