Abstract:Time series classification and forecasting have long been studied with the traditional statistical methods. Recently, deep learning achieved remarkable successes in areas such as image, text, video, audio processing, etc. However, research studies conducted with deep neural networks in these fields are not abundant. Therefore, in this paper, we aim to propose and evaluate several state-of-the-art neural network models in these fields. We first review the basics of representative models, namely long short-term memory and its variants, the temporal convolutional network and the generative adversarial network. Then, long short-term memory with autoencoder and attention-based models, the temporal convolutional network and the generative adversarial model are proposed and applied to time series classification and forecasting. Gaussian sliding window weights are proposed to speed the training process up. Finally, the performances of the proposed methods are assessed using five optimizers and loss functions with the public benchmark datasets, and comparisons between the proposed temporal convolutional network and several classical models are conducted. Experiments show the proposed models’ effectiveness and confirm that the temporal convolutional network is superior to long short-term memory models in sequence modeling. We conclude that the proposed temporal convolutional network reduces time consumption to around 80% compared to others while retaining the same accuracy. The unstable training process for generative adversarial network is circumvented by tuning hyperparameters and carefully choosing the appropriate optimizer of “Adam”. The proposed generative adversarial network also achieves comparable forecasting accuracy with traditional methods.

Theoretical analysis of deep neural networks for temporally dependent observations

Sparse Deep Learning for Time Series Data: Theory and Applications

Finite-Time Analysis of Adaptive Temporal Difference Learning with Deep Neural Networks

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

Dense ReLU Neural Networks for Temporal-spatial Model

Adaptive deep learning for nonlinear time series models

An Improved Finite-time Analysis of Temporal Difference Learning with Deep Neural Networks

Deep learning from strongly mixing observations: Sparse-penalized regularization and minimax optimality

Nonparametric regression for repeated measurements with deep neural networks

A Neural Networks Based Method for Multivariate Time-Series Forecasting

A Review of Deep Learning Models for Time Series Prediction

AdaRNN: Adaptive Learning and Forecasting of Time Series

Neural network stochastic differential equation models with applications to financial data forecasting

The Disappearance of Timestep Embedding in Modern Time-Dependent Neural Networks

Deep Process Neural Network for Temporal Deep Learning

Excess risk bound for deep learning under weak dependence

On Non-asymptotic Theory of Recurrent Neural Networks in Temporal Point Processes

Discovering long term dependencies in noisy time series data using deep learning

Time Series Forecasting and Classification Models Based on Recurrent with Attention Mechanism and Generative Adversarial Networks

Statistical Properties of Deep Neural Networks with Dependent Data

A novel hybrid deep learning time series forecasting model based on long-short-term patterns