Abstract:A delay dynamical system can fold a feedforward neural network into one nonlinear neuron and multiple delay loops under the non-Von Neumann structure, greatly decreasing the hardware requirements. In this paper, we transform the folded-in-time DNN (Fit-DNN) into a folded-in-time RNN (Fit-RNN) and derive the backpropagation algorithm for it. The performance of the folded reservoir computing (DRC), Fit-DNN, and Fit-RNN is compared on time-series prediction and classification tasks, respectively. The impact of virtual node separation on the performance of Fit-RNN is analyzed. The limits of Fit-RNN's capabilities in conducting effective time-series predictions were determined based on the NARMA-X series tasks. We calculate the 5-order information processing capacity (IPC) of DRC and Fit-RNN on NARMA10. The results indicate that Fit-RNN has almost the necessary information transformation ability for the task. In the recognition tasks conducted separately on spoken digits and speakers, we investigate the impact of the number and separation of virtual nodes on the recognition capability of the three folded networks. The results demonstrate that Fit-RNN shows more promise in handling long sequences and large-scale recognition tasks. The recognition accuracy is further enhanced by increasing the number of virtual nodes and node separation. Furthermore, the introduction of two common types of noise in the speaker recognition task environment further highlights the potential of Fit-RNN in practical applications. Furthermore, two common types of noise are introduced into the speaker recognition task, further highlighting the potential of Fit-RNN in practical applications.

Dropframe Scheme in Recurrent Neural Networks for Time Series Modeling

Residual Recurrent Neural Networks for Learning Sequential Representations.

Frame Prediction Using Recurrent Convolutional Encoder with Residual Learning

Frame Stacking and Retaining for Recurrent Neural Network Acoustic Model

Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks

A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

Deep RNN Framework for Visual Sequential Applications

DRRNets: Dynamic Recurrent Routing Via Low-Rank Regularization in Recurrent Neural Networks.

An Improved Time Feedforward Connections Recurrent Neural Networks

Hierarchically Gated Recurrent Neural Network for Sequence Modeling

Recurrent Neural Network Regularization

PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs

R-Drop: Regularized Dropout for Neural Networks.

Restricted Recurrent Neural Networks

Hierarchical Video Frame Sequence Representation with Deep Convolutional Graph Network

MS-RNN: A Flexible Multi-Scale Framework for Spatiotemporal Predictive Learning

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning

Learning various length dependence by dual recurrent neural networks

Deep Learning with Long Short-Term Memory for Time Series Prediction

Time Series Prediction Method Based on Variant LSTM Recurrent Neural Network

Single-layer Folded RNN for Time Series Prediction and Classification under a Non-Von Neumann Architecture