Abstract:Transformer-based models demonstrate tremendous potential for Multivariate Time Series (MTS) forecasting due to their ability to capture long-term temporal dependencies by using the self-attention mechanism. However, effectively modeling the spatial correlation cross series for MTS is a challenge for Transformer. Although Graph Neural Networks (GNN) are competent for modeling spatial dependencies across series, existing methods are based on the assumption of static relationships between variables, which do not align with the time-varying spatial dependencies in real-world series. Therefore, we propose DyGraphformer, which integrates graph convolution into Transformer to assist Transformer in effectively modeling spatial dependencies, while also dynamically inferring time-varying spatial dependencies by combining historical spatial information. In DyGraphformer, decoder module involving complex recursion is abandoned to accelerate model execution. First, the input is embedded using DSW (Dimension Segment Wise) through integrating its position and node level embedding to preserve temporal and spatial information. Then, the time self-attention layer and dynamic graph convolutional layer are constructed to capture temporal dependencies and spatial dependencies of multivariate time series, respectively. The dynamic graph convolutional layer utilizes Gated Recurrent Unit (GRU) to obtain historical spatial dependencies, and integrates the series features of the current time to perform graph structure inference in multiple subspaces. Specifically, to fully utilize the spatio-temporal information at different scales, DyGraphformer performs hierarchical encoder learning for the final forecasting. Extensive experimental results on seven real-world datasets demonstrate DyGraphformer outperforms state-of-the-art baseline methods, with comparisons including Transformer-based and GNN-based methods.

Deep transition network with gating mechanism for multivariate time series forecasting

Gated Transformer Networks for Multivariate Time Series Classification

Double-Path Adaptive-correlation Spatial-Temporal Inverted Transformer for Stock Time Series Forecasting

Deconstructing Recurrence, Attention, and Gating: Investigating the transferability of Transformers and Gated Recurrent Neural Networks in forecasting of dynamical systems

Gated Res2Net for Multivariate Time Series Analysis

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

A Hybrid Spiking Neurons Embedded LSTM Network for Multivariate Time Series Learning under Concept-Drift Environment

Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting

Adaptive Graph Convolutional Network Framework for Multidimensional Time Series Prediction

TDG4MSF: A temporal decomposition enhanced graph neural network for multivariate time series forecasting

DyGraphformer: Transformer combining dynamic spatio-temporal graph network for multivariate time series forecasting

STTD: spatial-temporal transformer with double recurrent graph convolutional cooperative network for traffic flow prediction

Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting

ADCT-Net: Adaptive traffic forecasting neural network via dual-graphic cross-fused transformer

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

AFSTGCN: Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network

A Multiscale Interactive Recurrent Network for Time-Series Forecasting

MTHetGNN: A Heterogeneous Graph Embedding Framework for Multivariate Time Series Forecasting

STGIN: A Spatial Temporal Graph-Informer Network for Long Sequence Traffic Speed Forecasting

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

Convolutional Neural Network-Based Bidirectional Gated Recurrent Unit–Additive Attention Mechanism Hybrid Deep Neural Networks for Short-Term Traffic Flow Prediction