Time Series Forecasting and Classification Models Based on Recurrent with Attention Mechanism and Generative Adversarial Networks

Kun Zhou,Wenyong Wang,Teng Hu,Kai Deng

DOI: https://doi.org/10.3390/s20247211

IF: 3.9

2020-12-16

Sensors

Abstract:Time series classification and forecasting have long been studied with the traditional statistical methods. Recently, deep learning achieved remarkable successes in areas such as image, text, video, audio processing, etc. However, research studies conducted with deep neural networks in these fields are not abundant. Therefore, in this paper, we aim to propose and evaluate several state-of-the-art neural network models in these fields. We first review the basics of representative models, namely long short-term memory and its variants, the temporal convolutional network and the generative adversarial network. Then, long short-term memory with autoencoder and attention-based models, the temporal convolutional network and the generative adversarial model are proposed and applied to time series classification and forecasting. Gaussian sliding window weights are proposed to speed the training process up. Finally, the performances of the proposed methods are assessed using five optimizers and loss functions with the public benchmark datasets, and comparisons between the proposed temporal convolutional network and several classical models are conducted. Experiments show the proposed models’ effectiveness and confirm that the temporal convolutional network is superior to long short-term memory models in sequence modeling. We conclude that the proposed temporal convolutional network reduces time consumption to around 80% compared to others while retaining the same accuracy. The unstable training process for generative adversarial network is circumvented by tuning hyperparameters and carefully choosing the appropriate optimizer of “Adam”. The proposed generative adversarial network also achieves comparable forecasting accuracy with traditional methods.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper primarily focuses on utilizing advanced Deep Neural Network (DNN) models for time series forecasting and classification tasks. Specifically, the objectives of the paper include the following aspects:

1. **Propose Improved Time Series Forecasting Models**:
- Propose a Long Short-Term Memory (LSTM) network combined with autoencoders and attention mechanisms, and apply it to time series forecasting modeling.
- Propose a Gaussian sliding window weight initialization method to accelerate the training process of the LSTM attention model.

2. **Evaluate Models Based on Temporal Convolutional Networks (TCN)**:
- Evaluate the performance of TCN in time series forecasting and classification tasks, and demonstrate its superiority over classical methods such as 1NN-DTW, BOSS, and WEASEL.
- Compare the performance of TCN and LSTM architectures in sequence modeling, verifying the efficiency advantage of TCN.

3. **Apply Generative Adversarial Networks (GAN) for Time Series Forecasting**:
- Propose a GAN model based on LSTM as the generator (G) and Multi-Layer Perceptron (MLP) as the discriminator (D) for time series forecasting tasks.
- Evaluate the performance of the proposed GAN model against traditional ARIMA models in time series forecasting and compare their accuracy.

In summary, the paper aims to improve existing methods for time series forecasting and classification by introducing new model architectures and techniques, and to validate the effectiveness and superiority of these new methods.