Abstract:Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is capturing (1) the spatial correlations at the same time, (2) the spatio-temporal relationships at different times, and (3) long-term dependency of the temporal relationships between different series. Attention-based recurrent neural networks (RNN) can effectively represent and learn the dynamic spatio-temporal relationships between exogenous series and target series, but they only perform well in one-step time prediction and short-term time prediction. In this paper, inspired by human attention mechanism including the dual-stage two-phase (DSTP) model and the influence mechanism of target information and non-target information, we propose DSTP-based RNN (DSTP-RNN) and DSTP-RNN-II respectively for long-term time series prediction. Specifically, we first propose the DSTP-based structure to enhance the spatial correlations between exogenous series. The first phase produces violent but decentralized response weight, while the second phase leads to stationary and concentrated response weight. Then, we employ multiple attentions on target series to boost the long-term dependency. Finally, we study the performance of deep spatial attention mechanism and provide interpretation. Experimental results demonstrate that the present work can be successfully used to develop expert or intelligent systems for a wide range of applications, with state-of-the-art performances superior to nine baseline methods on four datasets in the fields of energy, finance, environment and medicine, respectively. Overall, the present work carries a significant value not merely in the domain of machine intelligence and deep learning, but also in the fields of many applications. (C) 2019 Elsevier Ltd. All rights reserved.

Attention-mechanism Based DiPLS-LSTM and Its Application in Industrial Process Time Series Big Data Prediction

Enhanced Solar Power Prediction Using Attention-Based DiPLS-BiLSTM Model

Semi-supervised LSTM with Historical Feature Fusion Attention for Temporal Sequence Dynamic Modeling in Industrial Processes

Deep Learning with Spatiotemporal Attention-Based LSTM for Industrial Soft Sensor Model Development

Data-Based Line Trip Fault Prediction in Power Systems Using LSTM Networks and SVM.

A dual-stage attention-based Bi-LSTM network for multivariate time series prediction

A Multiphase Dual Attention-Based LSTM Neural Network for Industrial Product Quality Prediction

Spatio-temporal attention-based hybrid deep network for time series prediction of industrial process

A Deep Learning-Based Approach for High-Dimensional Industrial Steam Consumption Prediction to Enhance Sustainability Management

A Wide-Deep-Sequence Model-Based Quality Prediction Method in Industrial Process Analysis

A Deep Residual PLS for Data-Driven Quality Prediction Modeling in Industrial Process

Novel double-layer bidirectional LSTM network with improved attention mechanism for predicting energy consumption

RAdam-DA-NLSTM: A Nested LSTM-Based Time Series Prediction Method for Human–Computer Intelligent Systems

DTDR–ALSTM: Extracting dynamic time-delays to reconstruct multivariate data for improving attention-based LSTM industrial time series prediction models

Time Series Prediction Based on Time Attention Mechanism and LSTM Neural Network

Integrating spatio-positional series attention to deep network for multi-turbine short-term wind power prediction

DSTP-RNN: A Dual-Stage Two-Phase Attention-Based Recurrent Neural Network for Long-Term and Multivariate Time Series Prediction

Deep Learning with Long Short-Term Memory for Time Series Prediction

<i>A???????</i><SUP>2</SUP>-LSTM for predictive maintenance of industrial equipment based on machine learning

Semi-supervised Discriminative Projective Dictionary Pair Learning and Its Application to Industrial Process

Multi-level deep domain adaptive adversarial model based on tensor-train decomposition for industrial time series forecasting