Abstract:In recent years, spiking neural networks (SNNs), which originated from the theoretical basis of neuroscience, have attracted neuromorphic computing and brain-like computing due to their advantages, such as neural dynamics and coding mechanism, which are similar to biological neurons. SNNs have become one of the mainstream frameworks in the field of brain-like computing. However, most of the Leaky Integrate-and-Fire (LIF) neuron models currently used by SNNs based on direct training of backpropagation (BP) do not consider the changes in the recurrent connections and the dynamic strength of neuron connections over time. This study presented the LIF neuron model with recurrent connections and a method for dynamically changing the presynaptic currents. Recurrent LIF neurons have an additional cyclic connection compared with classic LIF neurons. Their postsynaptic current stimulates a change in membrane potential at the next time point. Their dynamics were more similar to the activities of biological neurons. We also proposed an efficient and flexible BP training method for recurrent LIF neurons. On the basis of the above methods, we proposed the recurrent SNN with dynamic presynaptic currents based on backpropagation (RDS-BP). We test the proposed RDS-BP on three image data sets (MNIST, Fashion-MNIST and CIFAR-10) and two text data sets (IMDB and TREC). The results showed that the performance of RDS-BP not only exceeded the naive SNN models based on BP but also exceeded the SNN methods proposed in previous studies in recent years, which had excellent performance in previous experiments. Our work provides a new LIF neuron model with a recurrent connection and dynamic presynaptic current and a BP training arrangement for the proposed neuron, which could merit developments with neuromorphic and brain-like computing.

A Study of a Class of Dynamical Recurrent Neural Networks Based on an Improved Algorithm

Recursive Prediction Error Algorithm of Recurrent Neural Networks and Its Application on Nonlinear Dynamic System Modelling

Recurrent Neural Networks for Recursive Identification of Nonlinear Dynamic Process

Modeling and Application Based on Diagonal Recurrent Neural Network

Dynamic T-S Recurrent Networks and Their Application

A study on time series prediction model based on CRBM algorithm

Design, Verification, and Application of New Discrete-Time Recurrent Neural Network for Dynamic Nonlinear Equations Solving

DRRNets: Dynamic Recurrent Routing Via Low-Rank Regularization in Recurrent Neural Networks.

Dynamic T-S Recurrent Fuzzy Neural Networks and Its Application

Model Predictive Control for Complex Systems Based on Dynamic Recurrent Neural Network

Reconstruction of a nonlinear dynamics via a dynamic recurrent neural network

A Generalized Cost Criterion Based Learning Algorithm For Diagonal Recurrent Neural Networks

An Improved Back-propagation NN Algorithm And Its Application

The Research and Application of Image Recognition Based on Improved BP Algorithm

Research And Simulation Of A Diagonal Recurrent Neural Network

Ship Motion Extreme Short Time Prediction of Ship Pitch Based on Diagonal Recurrent Neural Network

A Dynamic Model for Traffic Flow Prediction Using Improved DRN

Adaptive Learning with Guaranteed Stability for Discrete-Time Recurrent Neural Networks

Recurrent Spiking Neural Network with Dynamic Presynaptic Currents Based on Backpropagation

Research of a Diagonal Recurrent Neural Network and Artificial Neural Networks

Nonlinear Dynamical System Simulation Based on Recurrent Wavelet Neural Network