Abstract:Time series play an important role in many fields, such as industrial control, automated monitoring, and weather forecasting. Because there is often more than one variable in reality problems and they are related to each other, the multivariable time series (MTS) introduced. Using historical observations to accurately predict MTS is still very challenging. Therefore, a new time series prediction model proposed based on the temporal self-attention mechanism, convolutional neural network and long short-term memory (Conv-LSTM). When the standard attention mechanism for time series is combined with recurrent neural network (RNN), it heavily depends on the hidden state of the RNN. Particularly in the first time step, the initial hidden state (typically 0) must be artificially introduced to calculate the attention weight of that step, which results in additional noise in the calculation of the attention weight. To address this problem and increase the flexibility of the attention layer, a new self-attention mechanism designed to extract the temporal dependence of the MTS, which called temporal self-attention. In this attention mechanism, long short-term memory (LSTM) adopted as a sequence encoder to calculate the query, key, and value to obtain a more complete temporal dependence than standard self-attention. Because of flexibility of this structure, the DA-Conv-LSTM model was improved, in which a SOTA attention-based method used for MTS prediction. Our improved model compared with six baseline models on multiple datasets (SML2010 and NASDAQ100), and applied to satellite state prediction (our private dataset). The effectiveness of our temporal self-attention was demonstrated by experiments. And the best short-term prediction performance was achieved by our improved model.

TATCN: Time Series Prediction Model Based on Time Attention Mechanism and TCN

The Temperature Prediction of Hydro-generating Units based on Temporal Convolutional Network and Recurrent Neural Network

High-efficiency Chaotic Time Series Prediction Based on Time Convolution Neural Network

Parallel spatio-temporal attention-based TCN for multivariate time series prediction

Hierarchical Time Series Forecasting Based on Temporal Convolution and Attention Mechanism

Prediction Algorithm for Heat Demand of Science and Technology Topics Based on Time Convolution Network

Temporal Convolutional Networks Applied to Energy-related Time Series Forecasting

Temporal Convolutional Attention-based Network For Sequence Modeling

AMTCN: An Attention-Based Multivariate Temporal Convolutional Network for Electricity Consumption Prediction

Temporal Attention Based TCN-BIGRU Model for Energy Time Series Forecasting

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

Temporal Chain Network with Intuitive Attention Mechanism for Long-Term Series Forecasting.

Adapted Receptive Field Temporal Convolutional Networks with Bar-Shaped Structures Tailored to Industrial Process Operation Models

TCN-Attention-BIGRU: Building energy modelling based on attention mechanisms and temporal convolutional networks

DDTCN: Decomposed dimension time-domain convolutional neural network along spatial dimensions for multiple long-term series forecasting

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

Short-term power load forecasting based on TCN-LSTM model

TSCND: Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

TEC Prediction Based on Att-CNN-BiLSTM

Time Series Prediction Based on LSTM-Attention-LSTM Model

A novel hybrid framework based on temporal convolution network and transformer for network traffic prediction