Abstract:Spiking neural networks (SNNs) are positioned to enable spatio-temporal information processing and ultra-low power event-driven neuromorphic hardware. However, SNNs are yet to reach the same performances of conventional deep artificial neural networks (ANNs), a long-standing challenge due to complex dynamics and non-differentiable spike events encountered in training. The existing SNN error backpropagation (BP) methods are limited in terms of scalability, lack of proper handling of spiking discontinuities, and/or mismatch between the rate-coded loss function and computed gradient. We present a hybrid macro/micro level backpropagation (HM2-BP) algorithm for training multi-layer SNNs. The temporal effects are precisely captured by the proposed spike-train level post-synaptic potential (S-PSP) at the microscopic level. The rate-coded errors are defined at the macroscopic level, computed and back-propagated across both macroscopic and microscopic levels. Different from existing BP methods, HM2-BP directly computes the gradient of the rate-coded loss function w.r.t tunable parameters. We evaluate the proposed HM2-BP algorithm by training deep fully connected and convolutional SNNs based on the static MNIST [14] and dynamic neuromorphic N-MNIST [26]. HM2-BP achieves an accuracy level of 99.49% and 98.88% for MNIST and N-MNIST, respectively, outperforming the best reported performances obtained from the existing SNN BP algorithms. Furthermore, the HM2-BP produces the highest accuracies based on SNNs for the EMNIST [3] dataset, and leads to high recognition accuracy for the 16-speaker spoken English letters of TI46 Corpus [16], a challenging patio-temporal speech recognition benchmark for which no prior success based on SNNs was reported. It also achieves competitive performances surpassing those of conventional deep learning models when dealing with asynchronous spiking streams.

FP-MRBP: Fine-grained Parallel MapReduce Back Propagation Algorithm.

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Run-time prediction algorithm for parallel jobs in grid

BPPSA: Scaling Back-propagation by Parallel Scan Algorithm

Fuzzy Method and Neural Network Model Parallel Implementation of Multi-Layer Neural Network Based on Cloud Computing for Real Time Data Transmission in Large Offshore Platform

Unlocking Deep Learning: A BP-Free Approach for Parallel Block-Wise Training of Neural Networks

Brief Introduction of Back Propagation (BP) Neural Network Algorithm and Its Improvement

Research and Application on BP Neural Network Algorithm

FBP-Net for Direct Reconstruction of Dynamic PET Images.

2BP: 2-Stage Backpropagation

New and Rapidly Converging BP Neural Network Algorithm

A parallel compact Marine Predators Algorithm applied in time series prediction of Backpropagation neural network (BNN) and engineering optimization

Parallelization Implementation of Back Projection Algorithm Based on FPGA

An Integer-Fractional Gradient Algorithm for Back Propagation Neural Networks

Fast Learning Algorithms for Feedforward Neural Networks

Parallel-in-Time Solutions with Random Projection Neural Networks

Optimal parameters selection of back propagation algorithm in the feedforward neural network

LI-MR: A Local Iteration Map/Reduce Model and Its Application to Mine Community Structure in Large-Scale Networks

Optimization Study of BP Neural Network Based on Genetic Algorithm

Back Propagation With Randomized Cost Function For Training Neural Networks

Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks