Abstract:Transformer-based models demonstrate tremendous potential for Multivariate Time Series (MTS) forecasting due to their ability to capture long-term temporal dependencies by using the self-attention mechanism. However, effectively modeling the spatial correlation cross series for MTS is a challenge for Transformer. Although Graph Neural Networks (GNN) are competent for modeling spatial dependencies across series, existing methods are based on the assumption of static relationships between variables, which do not align with the time-varying spatial dependencies in real-world series. Therefore, we propose DyGraphformer, which integrates graph convolution into Transformer to assist Transformer in effectively modeling spatial dependencies, while also dynamically inferring time-varying spatial dependencies by combining historical spatial information. In DyGraphformer, decoder module involving complex recursion is abandoned to accelerate model execution. First, the input is embedded using DSW (Dimension Segment Wise) through integrating its position and node level embedding to preserve temporal and spatial information. Then, the time self-attention layer and dynamic graph convolutional layer are constructed to capture temporal dependencies and spatial dependencies of multivariate time series, respectively. The dynamic graph convolutional layer utilizes Gated Recurrent Unit (GRU) to obtain historical spatial dependencies, and integrates the series features of the current time to perform graph structure inference in multiple subspaces. Specifically, to fully utilize the spatio-temporal information at different scales, DyGraphformer performs hierarchical encoder learning for the final forecasting. Extensive experimental results on seven real-world datasets demonstrate DyGraphformer outperforms state-of-the-art baseline methods, with comparisons including Transformer-based and GNN-based methods.

Tigc-Net: Transformer-Improved Graph Convolution Network for Spatio-Temporal Prediction

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

Temporal Graph Transformer for Dynamic Network

STTG-net: a Spatio-temporal Network for Human Motion Prediction Based on Transformer and Graph Convolution Network

Spatial-Temporal Convolutional Transformer Network for Multivariate Time Series Forecasting

Spatio-Temporal Attention-based Graph Convolution Networks for Traffic Prediction

TVGCN: Time-varying graph convolutional networks for multivariate and multifeature spatiotemporal series prediction

Synchronous Spatiotemporal Graph Transformer: A New Framework for Traffic Data Prediction

TANGO: A temporal spatial dynamic graph model for event prediction

Biased Temporal Convolution Graph Network for Time Series Forecasting with Missing Values.

Multivariate long sequence time-series forecasting using dynamic graph learning

Dynamic spatial aware graph transformer for spatiotemporal traffic flow forecasting

TGN: A Temporal Graph Network for Physics Prediction

DyGraphformer: Transformer combining dynamic spatio-temporal graph network for multivariate time series forecasting

STGHTN: Spatial-temporal gated hybrid transformer network for traffic flow forecasting

Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction

FD-TGCN: Fast and dynamic temporal graph convolution network for traffic flow prediction

Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting

Spatio-Temporal Graph-TCN Neural Network for Traffic Flow Prediction

LSTGCN: Inductive Spatial Temporal Imputation Using Long Short Term Dependencies

Double-Path Adaptive-correlation Spatial-Temporal Inverted Transformer for Stock Time Series Forecasting