Abstract:Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is capturing (1) the spatial correlations at the same time, (2) the spatio-temporal relationships at different times, and (3) long-term dependency of the temporal relationships between different series. Attention-based recurrent neural networks (RNN) can effectively represent and learn the dynamic spatio-temporal relationships between exogenous series and target series, but they only perform well in one-step time prediction and short-term time prediction. In this paper, inspired by human attention mechanism including the dual-stage two-phase (DSTP) model and the influence mechanism of target information and non-target information, we propose DSTP-based RNN (DSTP-RNN) and DSTP-RNN-II respectively for long-term time series prediction. Specifically, we first propose the DSTP-based structure to enhance the spatial correlations between exogenous series. The first phase produces violent but decentralized response weight, while the second phase leads to stationary and concentrated response weight. Then, we employ multiple attentions on target series to boost the long-term dependency. Finally, we study the performance of deep spatial attention mechanism and provide interpretation. Experimental results demonstrate that the present work can be successfully used to develop expert or intelligent systems for a wide range of applications, with state-of-the-art performances superior to nine baseline methods on four datasets in the fields of energy, finance, environment and medicine, respectively. Overall, the present work carries a significant value not merely in the domain of machine intelligence and deep learning, but also in the fields of many applications. (C) 2019 Elsevier Ltd. All rights reserved.

Spatiotemporal Self-Attention-Based LSTNet for Multivariate Time Series Prediction

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation

DSTP-RNN: A Dual-Stage Two-Phase Attention-Based Recurrent Neural Network for Long-Term and Multivariate Time Series Prediction

DSTP-RNN: a dual-stage two-phase attention-based recurrent neural networks for long-term and multivariate time series prediction

Parallel spatio-temporal attention-based TCN for multivariate time series prediction

Self-attention-based time-variant neural networks for multi-step time series forecasting

Spatiotemporal Transformer Neural Network for Time-Series Forecasting

TSARNet :Integrated Spatio-Temporal Attention Mechanism for Multivariate Time Series Prediction Deep Learning Framework

Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction

A dual‐stage attention‐based Conv‐LSTM network for spatio‐temporal correlation and multivariate time series prediction

Multistage attention network for multivariate time series prediction

Time Series Prediction Based on LSTM-Attention-LSTM Model

A Hybrid Spiking Neurons Embedded LSTM Network for Multivariate Time Series Learning under Concept-Drift Environment

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

A Self-Attention Based Method for Wind Speed Forecasting in High-Speed Railway System

A Temporal Window Attention-Based Window-Dependent Long Short-Term Memory Network for Multivariate Time Series Prediction.

Predicting High-Dimensional Time Series Data with Spatial, Temporal and Global Information

Wavelet-Temporal Neural Network for Multivariate Time Series Prediction

Attention Based CNN-LSTM Network for Anomaly Pattern Classification of Multivariate Time Series

STDNet: A Spatio-Temporal Decomposition Neural Network for Multivariate Time Series Forecasting