Abstract:Time series play an important role in many fields, such as industrial control, automated monitoring, and weather forecasting. Because there is often more than one variable in reality problems and they are related to each other, the multivariable time series (MTS) introduced. Using historical observations to accurately predict MTS is still very challenging. Therefore, a new time series prediction model proposed based on the temporal self-attention mechanism, convolutional neural network and long short-term memory (Conv-LSTM). When the standard attention mechanism for time series is combined with recurrent neural network (RNN), it heavily depends on the hidden state of the RNN. Particularly in the first time step, the initial hidden state (typically 0) must be artificially introduced to calculate the attention weight of that step, which results in additional noise in the calculation of the attention weight. To address this problem and increase the flexibility of the attention layer, a new self-attention mechanism designed to extract the temporal dependence of the MTS, which called temporal self-attention. In this attention mechanism, long short-term memory (LSTM) adopted as a sequence encoder to calculate the query, key, and value to obtain a more complete temporal dependence than standard self-attention. Because of flexibility of this structure, the DA-Conv-LSTM model was improved, in which a SOTA attention-based method used for MTS prediction. Our improved model compared with six baseline models on multiple datasets (SML2010 and NASDAQ100), and applied to satellite state prediction (our private dataset). The effectiveness of our temporal self-attention was demonstrated by experiments. And the best short-term prediction performance was achieved by our improved model.

Spatiotemporal Learning with Decoupled Causal Attention for Multivariate Time Series

Causal augmented ConvNet: A temporal memory dilated convolution model for long-sequence time series prediction

Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation

Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning

Multivariate Time Series Forecasting with Causal-Temporal Attention Network.

Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction

Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling

A Multivariate Time Series Prediction Method Based on Convolution-Residual Gated Recurrent Neural Network and Double-Layer Attention

STCA: Spatio-Temporal Credit Assignment with Delayed Feedback in Deep Spiking Neural Networks

Spatiotemporal Self-Attention-Based LSTNet for Multivariate Time Series Prediction

Attention-Based Deep Spiking Neural Networks for Temporal Credit Assignment Problems.

Deep Coupling Network For Multivariate Time Series Forecasting

Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation

Multistage attention network for multivariate time series prediction

Self-Attention Causal Dilated Convolutional Neural Network for Multivariate Time Series Classification and Its Application

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

A dual-stage attention-based Bi-LSTM network for multivariate time series prediction

Is Single Enough? A Joint Spatiotemporal Feature Learning Framework for Multivariate Time Series Prediction

Learning Decomposed Spatial Relations for Multi-Variate Time-Series Modeling.