Abstract:The spiking neural networks (SNNs) are the third generation of neural networks and perform remarkably well in cognitive tasks such as pattern recognition. The spike emitting and information processing mechanisms found in biological cognitive systems motivate the application of the hierarchical structure and temporal encoding mechanism in spiking neural networks, which have exhibited strong computational capability. However, the hierarchical structure and temporal encoding approach require neurons to process information serially in space and time respectively, which reduce the training efficiency significantly. For training the hierarchical SNNs, most existing methods are based on the traditional back-propagation algorithm, inheriting its drawbacks of the gradient diffusion and the sensitivity on parameters. To keep the powerful computation capability of the hierarchical structure and temporal encoding mechanism, but to overcome the low efficiency of the existing algorithms, a new training algorithm, the Normalized Spiking Error Back Propagation (NSEBP) is proposed in this paper. In the feedforward calculation, the output spike times are calculated by solving the quadratic function in the spike response model instead of detecting postsynaptic voltage states at all time points in traditional algorithms. Besides, in the feedback weight modification, the computational error is propagated to previous layers by the presynaptic spike jitter instead of the gradient decent rule, which realizes the layer-wised training. Furthermore, our algorithm investigates the mathematical relation between the weight variation and voltage error change, which makes the normalization in the weight modification applicable. Adopting these strategies, our algorithm outperforms the traditional SNN multi-layer algorithms in terms of learning efficiency and parameter sensitivity, that are also demonstrated by the comprehensive experimental results in this paper.

Mlpnn Training Via A Multiobjective Optimization of Training Error and Stochastic Sensitivity

A Neural Network Model For General Minimax Problem

Neural-network-based Optimization and Analysis for Nonlinear Stochastic Systems.

Robust Neural Networks Learning Via a Minimization of Stochastic Output Sensitivity

A Smooth Optimisation Perspective on Designing and Training Feedforward Multilayer Perceptrons

Multilayer Perceptron optimization through Simulated Annealing and Fast Simulated Annealing

The Balanced Multi-Modal Spiking Neural Networks with Online Loss Adjustment and Time Alignment

Adaptive Smoothing Gradient Learning for Spiking Neural Networks.

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Convergence and objective functions of noise-injected multilayer perceptrons with hidden multipliers

Training Multi-Layer Perceptron with Enhanced Brain Storm Optimization Metaheuristics

Controlling Model Complexity in Probabilistic Model-Based Dynamic Optimization of Neural Network Structures

An Efficient Supervised Training Algorithm for Multilayer Spiking Neural Networks

Smoothing Neural Network for Constrained Non-Lipschitz Optimization with Applications

Experimental Study on Error Functions for Multilayer Perceptron Neural Network Architecture Selection with Average Weighted F-Score Evaluation

Multilayer Perceptron Network Optimization for Chaotic Time Series Modeling.

Multi-Objective Linear Ensembles for Robust and Sparse Training of Few-Bit Neural Networks

A novel Mathematical Modeling for Deep Multilayer Perceptron Optimization: Architecture Optimization and Activation Functions Selection

The Multiscale Structure of Neural Network Loss Functions: The Effect on Optimization and Origin

Multilayer Photonic Spiking Neural Networks: Generalized Supervised Learning Algorithm and Network Optimization

Multi-Objective Evolutionary Optimization of Probabilistic Neural Network