Abstract:In this paper we present a new framework for time-series modeling that combines the best of traditional statistical models and neural networks. We focus on time-series with long-range dependencies, needed for monitoring fine granularity data (e.g. minutes, seconds, milliseconds), prevalent in operational use-cases. Traditional models, such as auto-regression fitted with least squares (Classic-AR) can model time-series with a concise and interpretable model. When dealing with long-range dependencies, Classic-AR models can become intractably slow to fit for large data. Recently, sequence-to-sequence models, such as Recurrent Neural Networks, which were originally intended for natural language processing, have become popular for time-series. However, they can be overly complex for typical time-series data and lack interpretability. A scalable and interpretable model is needed to bridge the statistical and deep learning-based approaches. As a first step towards this goal, we propose modelling AR-process dynamics using a feed-forward neural network approach, termed AR-Net. We show that AR-Net is as interpretable as Classic-AR but also scales to long-range dependencies. Our results lead to three major conclusions: First, AR-Net learns identical AR-coefficients as Classic-AR, thus being equally interpretable. Second, the computational complexity with respect to the order of the AR process, is linear for AR-Net as compared to a quadratic for Classic-AR. This makes it possible to model long-range dependencies within fine granularity data. Third, by introducing regularization, AR-Net automatically selects and learns sparse AR-coefficients. This eliminates the need to know the exact order of the AR-process and allows to learn sparse weights for a model with long-range dependencies.

Exploiting the interpretability and forecasting ability of the RBF-AR model for nonlinear time series

Time Series Forecasting Using a Hybrid RBF Neural Network and AR Model Based on Binomial Smoothing

A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling

Time Series Forecasting With Rbf Neural Network

Stability Analysis of Deep Belief Network: Based SD-AR Model for Nonlinear Time Series

A study on time series prediction model based on CRBM algorithm

A Hybrid Modeling Method Based on Linear AR and Nonlinear DBN-AR Model for Time Series Forecasting

Time Series Forecasting Model Using A Hybrid Arima And Neural Network

RBM-based Back Propagation Neural Network with BSASA Optimization for Time Series Forecasting

Autoregressive Modeling of Forest Dynamics

A Novel FRBF-Type Model for Nonlinear Time Series Prediction

Model selection for RBF-ARX models

AR-Net: A simple Auto-Regressive Neural Network for time-series

A novel expectation–maximization-based separable algorithm for parameter identification of RBF-AR model

Recurrent Support Vector Regression for a Non-Linear ARMA Model with Applications to Forecasting Financial Returns

Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting

Nonlinear time series modeling and prediction using RBF network with improved clustering algorithm

Hierarchical Estimation Approach for RBF-AR Models with Regression Weights Based on the Increasing Data Length

DBN based SD-ARX model for nonlinear time series prediction and analysis

Parameter estimation for a class of radial basis function-based nonlinear time-series models with moving average noises

Application of RBF Neural Network in Medium and Long-Term Load Forecasting