Abstract:The spiking neural networks (SNNs) are the third generation of neural networks and perform remarkably well in cognitive tasks such as pattern recognition. The spike emitting and information processing mechanisms found in biological cognitive systems motivate the application of the hierarchical structure and temporal encoding mechanism in spiking neural networks, which have exhibited strong computational capability. However, the hierarchical structure and temporal encoding approach require neurons to process information serially in space and time respectively, which reduce the training efficiency significantly. For training the hierarchical SNNs, most existing methods are based on the traditional back-propagation algorithm, inheriting its drawbacks of the gradient diffusion and the sensitivity on parameters. To keep the powerful computation capability of the hierarchical structure and temporal encoding mechanism, but to overcome the low efficiency of the existing algorithms, a new training algorithm, the Normalized Spiking Error Back Propagation (NSEBP) is proposed in this paper. In the feedforward calculation, the output spike times are calculated by solving the quadratic function in the spike response model instead of detecting postsynaptic voltage states at all time points in traditional algorithms. Besides, in the feedback weight modification, the computational error is propagated to previous layers by the presynaptic spike jitter instead of the gradient decent rule, which realizes the layer-wised training. Furthermore, our algorithm investigates the mathematical relation between the weight variation and voltage error change, which makes the normalization in the weight modification applicable. Adopting these strategies, our algorithm outperforms the traditional SNN multi-layer algorithms in terms of learning efficiency and parameter sensitivity, that are also demonstrated by the comprehensive experimental results in this paper.

Effective Transfer Learning Algorithm in Spiking Neural Networks

Adaptive Deep Spiking Neural Network with Global-Local Learning Via Balanced Excitatory and Inhibitory Mechanism

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Effective Active Learning Method for Spiking Neural Networks.

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

An Efficient Knowledge Transfer Strategy for Spiking Neural Networks from Static to Event Domain

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Biologically Inspired Structure Learning with Reverse Knowledge Distillation for Spiking Neural Networks

Spiking Deep Residual Networks.

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

Learnable Surrogate Gradient for Direct Training Spiking Neural Networks

LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networks

Direct Training for Spiking Neural Networks: Faster, Larger, Better

Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillation

Toward Scalable, Efficient, and Accurate Deep Spiking Neural Networks With Backward Residual Connections, Stochastic Softmax, and Hybridization

Benchmarking Spiking Neural Network Learning Methods with Varying Locality

Training Stronger Spiking Neural Networks with Biomimetic Adaptive Internal Association Neurons

An Efficient Supervised Training Algorithm for Multilayer Spiking Neural Networks

Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization