Abstract:Abstract As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for pro cessing sequential data like audio and text. However, most RSNNs suffer from two problems. First, due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. Second, training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNN, skip-connected self-recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs, while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method, backpropagated intrinsic plasticity (BIP), to boost the performance of ScSr-SNNs further by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Based on challenging speech, neuromorphic speech, and neuromorphic image data sets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.

Design of a Self-Organizing Recurrent RBF Neural Network Based on Spiking Mechanism

Splicing System Based Genetic Algorithms for Developing Rbf Networks Models

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

A new recursive least squares-based learning algorithm for spiking neurons

BP-SRM: A directly training algorithm for spiking neural network constructed by spike response model

Discussion of Stability in a Class of Models on Recurrent Radial Basis Function Neural Networks

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

A Spiking Neural Network System for Robust Sequence Recognition

Incorporating structural plasticity into self-organization recurrent networks for sequence learning

Design of a self-organizing reciprocal modular neural network for nonlinear system modeling

Nonlinear time series modeling and prediction using RBF network with improved clustering algorithm

The Growing Radial Basis Function (RBF) Neural Network and Its Applications.

Research and application of RBF neural network based on modified Levenberg-Marquardt

Self-tuning and approximation via RBF neural networks

First Error-Based Supervised Learning Algorithm for Spiking Neural Networks

Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

A Novel Two Stage Algorithm for Construction of RBF Neural Models Based on A-optimality Criterion

Deep Spiking Neural Network with Neural Oscillation and Spike-Phase Information.

Monitor-Based Spiking Recurrent Network for the Representation of Complex Dynamic Patterns

A Spiking Neural Network Structure Implementing Reinforcement Learning

Learning Algorithms for RBF Functions and Subspace Based Functions