Abstract:Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is capturing (1) the spatial correlations at the same time, (2) the spatio-temporal relationships at different times, and (3) long-term dependency of the temporal relationships between different series. Attention-based recurrent neural networks (RNN) can effectively represent and learn the dynamic spatio-temporal relationships between exogenous series and target series, but they only perform well in one-step time prediction and short-term time prediction. In this paper, inspired by human attention mechanism including the dual-stage two-phase (DSTP) model and the influence mechanism of target information and non-target information, we propose DSTP-based RNN (DSTP-RNN) and DSTP-RNN-II respectively for long-term time series prediction. Specifically, we first propose the DSTP-based structure to enhance the spatial correlations between exogenous series. The first phase produces violent but decentralized response weight, while the second phase leads to stationary and concentrated response weight. Then, we employ multiple attentions on target series to boost the long-term dependency. Finally, we study the performance of deep spatial attention mechanism and provide interpretation. Experimental results demonstrate that the present work can be successfully used to develop expert or intelligent systems for a wide range of applications, with state-of-the-art performances superior to nine baseline methods on four datasets in the fields of energy, finance, environment and medicine, respectively. Overall, the present work carries a significant value not merely in the domain of machine intelligence and deep learning, but also in the fields of many applications. (C) 2019 Elsevier Ltd. All rights reserved.

A Multivariate Time Series Prediction Method Based on Convolution-Residual Gated Recurrent Neural Network and Double-Layer Attention

Causal augmented ConvNet: A temporal memory dilated convolution model for long-sequence time series prediction

The Temperature Prediction of Hydro-generating Units based on Temporal Convolutional Network and Recurrent Neural Network

CDA-LSTM: an Evolutionary Convolution-Based Dual-Attention LSTM for Univariate Time Series Prediction

A dual‐stage attention‐based Conv‐LSTM network for spatio‐temporal correlation and multivariate time series prediction

LHCnn - A Novel Efficient Multivariate Time Series Prediction Framework Utilizing Convolutional Neural Networks.

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

Multistage attention network for multivariate time series prediction

A dual-stage attention-based Bi-LSTM network for multivariate time series prediction

Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction

Multiple convolutional neural networks for multivariate time series prediction

Wavelet-Temporal Neural Network for Multivariate Time Series Prediction

A Neural Networks Based Method for Multivariate Time-Series Forecasting

Novel hybrid multi-head self-attention and multifractal algorithm for non-stationary time series prediction

Research on Multi-Modal Time Series Data Prediction Method Based on Dual-Stage Attention Mechanism.

Spatiotemporal Self-Attention-Based LSTNet for Multivariate Time Series Prediction

DSTP-RNN: A Dual-Stage Two-Phase Attention-Based Recurrent Neural Network for Long-Term and Multivariate Time Series Prediction

GRAformer: A Gated Residual Attention Transformer for Multivariate Time Series Forecasting

Time Series Prediction Based on Temporal Convolutional Network

Sequence Attention for Multivariate Time Series Forecasting