Abstract:The spiking neural networks (SNNs) are the third generation of neural networks and perform remarkably well in cognitive tasks such as pattern recognition. The spike emitting and information processing mechanisms found in biological cognitive systems motivate the application of the hierarchical structure and temporal encoding mechanism in spiking neural networks, which have exhibited strong computational capability. However, the hierarchical structure and temporal encoding approach require neurons to process information serially in space and time respectively, which reduce the training efficiency significantly. For training the hierarchical SNNs, most existing methods are based on the traditional back-propagation algorithm, inheriting its drawbacks of the gradient diffusion and the sensitivity on parameters. To keep the powerful computation capability of the hierarchical structure and temporal encoding mechanism, but to overcome the low efficiency of the existing algorithms, a new training algorithm, the Normalized Spiking Error Back Propagation (NSEBP) is proposed in this paper. In the feedforward calculation, the output spike times are calculated by solving the quadratic function in the spike response model instead of detecting postsynaptic voltage states at all time points in traditional algorithms. Besides, in the feedback weight modification, the computational error is propagated to previous layers by the presynaptic spike jitter instead of the gradient decent rule, which realizes the layer-wised training. Furthermore, our algorithm investigates the mathematical relation between the weight variation and voltage error change, which makes the normalization in the weight modification applicable. Adopting these strategies, our algorithm outperforms the traditional SNN multi-layer algorithms in terms of learning efficiency and parameter sensitivity, that are also demonstrated by the comprehensive experimental results in this paper.

Winsorized dendritic neuron model artificial neural network and a robust training algorithm with Tukey’s biweight loss function based on particle swarm optimization

Effective Neural Network Training with a New Weighting Mechanism-Based Optimization Algorithm.

A Dendritic Neuron Model Optimized by Meta-Heuristics with a Power-Law-Distributed Population Interaction Network for Financial Time-Series Forecasting

A probabilistic learning algorithm for robust modeling using neural networks with random weights

Statistical learning algorithms for dendritic neuron model artificial neural network based on sine cosine algorithm

A New Robust Learning Algorithm for Ordinary Differential Equation Network

Multiplicative Reweighting for Robust Neural Network Optimization

Training simple recurrent deep artificial neural network for forecasting using particle swarm optimization

A multi-in and multi-out dendritic neuron model and its optimization

A new hybrid recurrent artificial neural network for time series forecasting

Bio-Inspired Adaptive Neurons for Dynamic Weighting in Artificial Neural Networks

Optimizing the echo state network with a binary particle swarm optimization algorithm

A Particle Swarm Optimization-Based Interpretable Spiking Neural Classifier with Time-Varying Weights

Design of Financial Management Model Using the Forward Neural Network Based on Particle Swarm Optimization Algorithm

An Efficient Supervised Training Algorithm for Multilayer Spiking Neural Networks

An Approximate Bayesian Long Short-Term Memory Algorithm for Outlier Detection

The Subjected Spds Algorithm Of Forward Neural Network

Robust Stochastic Configuration Networks with Kernel Density Estimation for Uncertain Data Regression

Optimizing connection weights in neural networks using the whale optimization algorithm

Empirical strategy for stretching probability distribution in neural-network-based regression

A Fast Precise-Spike And Weight-Comparison Based Learning Approach For Evolving Spiking Neural Networks