Abstract:A delay dynamical system can fold a feedforward neural network into one nonlinear neuron and multiple delay loops under the non-Von Neumann structure, greatly decreasing the hardware requirements. In this paper, we transform the folded-in-time DNN (Fit-DNN) into a folded-in-time RNN (Fit-RNN) and derive the backpropagation algorithm for it. The performance of the folded reservoir computing (DRC), Fit-DNN, and Fit-RNN is compared on time-series prediction and classification tasks, respectively. The impact of virtual node separation on the performance of Fit-RNN is analyzed. The limits of Fit-RNN's capabilities in conducting effective time-series predictions were determined based on the NARMA-X series tasks. We calculate the 5-order information processing capacity (IPC) of DRC and Fit-RNN on NARMA10. The results indicate that Fit-RNN has almost the necessary information transformation ability for the task. In the recognition tasks conducted separately on spoken digits and speakers, we investigate the impact of the number and separation of virtual nodes on the recognition capability of the three folded networks. The results demonstrate that Fit-RNN shows more promise in handling long sequences and large-scale recognition tasks. The recognition accuracy is further enhanced by increasing the number of virtual nodes and node separation. Furthermore, the introduction of two common types of noise in the speaker recognition task environment further highlights the potential of Fit-RNN in practical applications. Furthermore, two common types of noise are introduced into the speaker recognition task, further highlighting the potential of Fit-RNN in practical applications.

A Clockwork RNN

Residual Recurrent Neural Networks for Learning Sequential Representations.

Single-layer Folded RNN for Time Series Prediction and Classification under a Non-Von Neumann Architecture

A Recurrent Neural Network for Rhythmic Timing

LHCnn - A Novel Efficient Multivariate Time Series Prediction Framework Utilizing Convolutional Neural Networks.

Recurrently Controlled Recurrent Networks

PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning

Network Security Situation Prediction Based on Optimized Clock-Cycle Recurrent Neural Network for Sensor-Enabled Networks

Complex Recurrent Spectral Network

Recurrent Convolutional Neural Network for Sequential Recommendation.

Recurrent neural networks as neuro-computational models of human speech recognition

What to Do Next: Modeling User Behaviors by Time-LSTM

RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks

Dynamical system simulation with attention and recurrent neural networks

Fast-Slow Recurrent Neural Networks

Linear-Time Sequence Classification using Restricted Boltzmann Machines

Learning to (Learn at Test Time): RNNs with Expressive Hidden States

A Critical Review of Recurrent Neural Networks for Sequence Learning

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

Deep Learning with Long Short-Term Memory for Time Series Prediction

A survey on long short-term memory networks for time series prediction