Markovian RNN: An Adaptive Time Series Prediction Network With HMM-Based Switching for Nonstationary Environments
Fatih Ilhan,Oguzhan Karaahmetoglu,Ismail Balaban,Suleyman Serdar Kozat
DOI: https://doi.org/10.1109/tnnls.2021.3100528
IF: 14.255
2021-01-01
IEEE Transactions on Neural Networks and Learning Systems
Abstract:We investigate nonlinear regression for nonstationary sequential data. In most real-life applications such as business domains including finance, retail, energy, and economy, time series data exhibit nonstationarity due to the temporally varying dynamics of the underlying system. We introduce a novel recurrent neural network (RNN) architecture, which adaptively switches between internal regimes in a Markovian way to model the nonstationary nature of the given data. Our model, Markovian RNN employs a hidden Markov model (HMM) for regime transitions, where each regime controls hidden state transitions of the recurrent cell independently. We jointly optimize the whole network in an end-to-end fashion. We demonstrate the significant performance gains compared to conventional methods such as Markov Switching ARIMA, RNN variants and recent statistical and deep learning-based methods through an extensive set of experiments with synthetic and real-life datasets. We also interpret the inferred parameters and regime belief values to analyze the underlying dynamics of the given sequences.
computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture