Abstract:Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is capturing (1) the spatial correlations at the same time, (2) the spatio-temporal relationships at different times, and (3) long-term dependency of the temporal relationships between different series. Attention-based recurrent neural networks (RNN) can effectively represent and learn the dynamic spatio-temporal relationships between exogenous series and target series, but they only perform well in one-step time prediction and short-term time prediction. In this paper, inspired by human attention mechanism including the dual-stage two-phase (DSTP) model and the influence mechanism of target information and non-target information, we propose DSTP-based RNN (DSTP-RNN) and DSTP-RNN-II respectively for long-term time series prediction. Specifically, we first propose the DSTP-based structure to enhance the spatial correlations between exogenous series. The first phase produces violent but decentralized response weight, while the second phase leads to stationary and concentrated response weight. Then, we employ multiple attentions on target series to boost the long-term dependency. Finally, we study the performance of deep spatial attention mechanism and provide interpretation. Experimental results demonstrate that the present work can be successfully used to develop expert or intelligent systems for a wide range of applications, with state-of-the-art performances superior to nine baseline methods on four datasets in the fields of energy, finance, environment and medicine, respectively. Overall, the present work carries a significant value not merely in the domain of machine intelligence and deep learning, but also in the fields of many applications. (C) 2019 Elsevier Ltd. All rights reserved.

TSARNet :Integrated Spatio-Temporal Attention Mechanism for Multivariate Time Series Prediction Deep Learning Framework

Multi Time Scale Inception-Time Network for Soft Sensor of Blast Furnace Ironmaking Process

Parallel spatio-temporal attention-based TCN for multivariate time series prediction

Temporal pattern attention for multivariate time series forecasting

TS2ARCformer: A Multi-Dimensional Time Series Forecasting Framework for Short-Term Load Prediction

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

Spatiotemporal Self-Attention-Based LSTNet for Multivariate Time Series Prediction

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

Multi-scale adaptive attention-based time-variant neural networks for multi-step time series forecasting

Parallel Extraction of Long-term Trends and Short-term Fluctuation Framework for Multivariate Time Series Forecasting

Sequence Attention for Multivariate Time Series Forecasting

A Temporal Window Attention-Based Window-Dependent Long Short-Term Memory Network for Multivariate Time Series Prediction.

Temporal Chain Network with Intuitive Attention Mechanism for Long-Term Series Forecasting.

Temporal Attention Based TCN-BIGRU Model for Energy Time Series Forecasting

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

AdaRNN: Adaptive Learning and Forecasting of Time Series

Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction

TLNets: Transformation Learning Networks for long-range time-series prediction

DSTP-RNN: A Dual-Stage Two-Phase Attention-Based Recurrent Neural Network for Long-Term and Multivariate Time Series Prediction

Expressing Multivariate Time Series as Graphs with Time Series Attention Transformer