Abstract:Time series play an important role in many fields, such as industrial control, automated monitoring, and weather forecasting. Because there is often more than one variable in reality problems and they are related to each other, the multivariable time series (MTS) introduced. Using historical observations to accurately predict MTS is still very challenging. Therefore, a new time series prediction model proposed based on the temporal self-attention mechanism, convolutional neural network and long short-term memory (Conv-LSTM). When the standard attention mechanism for time series is combined with recurrent neural network (RNN), it heavily depends on the hidden state of the RNN. Particularly in the first time step, the initial hidden state (typically 0) must be artificially introduced to calculate the attention weight of that step, which results in additional noise in the calculation of the attention weight. To address this problem and increase the flexibility of the attention layer, a new self-attention mechanism designed to extract the temporal dependence of the MTS, which called temporal self-attention. In this attention mechanism, long short-term memory (LSTM) adopted as a sequence encoder to calculate the query, key, and value to obtain a more complete temporal dependence than standard self-attention. Because of flexibility of this structure, the DA-Conv-LSTM model was improved, in which a SOTA attention-based method used for MTS prediction. Our improved model compared with six baseline models on multiple datasets (SML2010 and NASDAQ100), and applied to satellite state prediction (our private dataset). The effectiveness of our temporal self-attention was demonstrated by experiments. And the best short-term prediction performance was achieved by our improved model.

AT-LSTM: An Attention-based LSTM Model for Financial Time Series Prediction

Attention-Ordering LSTM for Financial Time Series Forecasting

A Hybrid Attention-Based EMD-LSTM Model for Financial Time Series Prediction

A new attention-based LSTM model for closing stock price prediction

Time Series Prediction Based on LSTM-Attention-LSTM Model

EA-LSTM: Evolutionary attention-based LSTM for time series prediction

Financial series prediction using Attention LSTM

Research on financial assets transaction prediction model based on LSTM neural network

Real-time Forecasting of Time Series in Financial Markets Using Sequentially Trained Many-to-one LSTMs

Adaptability of Financial Time Series Prediction Based on BiLSTM

Movement Forecasting of Financial Time Series Based on Adaptive LSTM-BN Network.

Long Time Series Deep Forecasting with Multiscale Feature Extraction and Seq2seq Attention Mechanism

Effective LSTMs with Seasonal-Trend Decomposition and Adaptive Learning and Niching-Based Backtracking Search Algorithm for Time Series Forecasting

Exploring the attention mechanism in LSTM-based Hong Kong stock price movement prediction

Sports match prediction model for training and exercise using attention-based LSTM network

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

A dual-stage attention-based Bi-LSTM network for multivariate time series prediction

A Comparison of LSTMs and Attention Mechanisms for Forecasting Financial Time Series

Bi-LSTM Price Prediction based on Attention Mechanism

A novel attLSTM framework combining the attention mechanism and bidirectional LSTM for demand forecasting

Application of SDAE-LSTM Model on Financial Time Series Forecasting