Abstract:Time series play an important role in many fields, such as industrial control, automated monitoring, and weather forecasting. Because there is often more than one variable in reality problems and they are related to each other, the multivariable time series (MTS) introduced. Using historical observations to accurately predict MTS is still very challenging. Therefore, a new time series prediction model proposed based on the temporal self-attention mechanism, convolutional neural network and long short-term memory (Conv-LSTM). When the standard attention mechanism for time series is combined with recurrent neural network (RNN), it heavily depends on the hidden state of the RNN. Particularly in the first time step, the initial hidden state (typically 0) must be artificially introduced to calculate the attention weight of that step, which results in additional noise in the calculation of the attention weight. To address this problem and increase the flexibility of the attention layer, a new self-attention mechanism designed to extract the temporal dependence of the MTS, which called temporal self-attention. In this attention mechanism, long short-term memory (LSTM) adopted as a sequence encoder to calculate the query, key, and value to obtain a more complete temporal dependence than standard self-attention. Because of flexibility of this structure, the DA-Conv-LSTM model was improved, in which a SOTA attention-based method used for MTS prediction. Our improved model compared with six baseline models on multiple datasets (SML2010 and NASDAQ100), and applied to satellite state prediction (our private dataset). The effectiveness of our temporal self-attention was demonstrated by experiments. And the best short-term prediction performance was achieved by our improved model.

Compound Autoregressive Network for Prediction of Multivariate Time Series

Prediction of Multivariate Time Series Based on Principal Component Analysis and Neural Networks

Predicting Multivariate Nonlinear Time Series Based On A New Hybrid Neural Network

A Neural Networks Based Method for Multivariate Time-Series Forecasting

Wavelet-Temporal Neural Network for Multivariate Time Series Prediction

Application of Neural Networks on Multivariate Time Series Modeling and Prediction

TSARNet :Integrated Spatio-Temporal Attention Mechanism for Multivariate Time Series Prediction Deep Learning Framework

Multivariate Time Series Prediction Using a Hybridization of VARMA Models and Bayesian Networks

Multiple convolutional neural networks for multivariate time series prediction

Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction

Study of nonlinear multivariate time series prediction based on neural networks

Multivariate Nonlinear Time Series Prediction by Chain-Structure Echo State Network

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

Multivariate time series forecasting method based on nonlinear spiking neural P systems and non-subsampled shearlet transform

Multivariate time series modeling and prediction based on reservoir independent components

High-Order Temporal Correlation Model Learning for Time-Series Prediction.

Analysis and modeling of multivariate chaotic time series based on neural network

A Memory-Network Based Solution for Multivariate Time-Series Forecasting

Parallel Extraction of Long-term Trends and Short-term Fluctuation Framework for Multivariate Time Series Forecasting

Industrial equipment structure multivariate regression prediction via random input and hybrid temporal neural networks