Abstract:Spiking Neural Networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. The challenge of spike-based encoding has been the lack of a universal training mechanism performed entirely using spikes. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANN): (1) SNNs can be trained by externally computed numerical gradients. (2) A major advancement towards native spike-based learning has been the use of approximate Backpropagation using spike-time dependent plasticity (STDP) with phased forward/backward passes. However, the transfer of information between such phases for gradient and weight update calculation necessitates external memory and computational access. This is a challenge for standard neuromorphic hardware implementations. In this paper, we propose a stochastic SNN based Back-Prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly with spikes. Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams. The composite neuron encodes information in the form of stochastic spike-trains and converts Backpropagation weight updates into temporally and spatially local spike coincidence updates compatible with hardware-friendly Resistive Processing Units (RPUs). Furthermore, we characterize the quantization effect of discrete spike-based weight update to show that our method approaches BP ANN baseline with sufficiently long spike-trains. Finally, we show that the well-performing softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a Winner Take All (WTA) rule. Our SNN with a 2-layer network shows excellent generalization through comparable performance to ANNs with equivalent architecture and regularization parameters on static image datasets like MNIST, Fashion-MNIST, Extended MNIST, and temporally encoded image datasets like Neuromorphic MNIST datasets. Thus, SSNN-BP enables BP compatible with purely spike-based neuromorphic hardware.

Learning and real-time classification of hand-written digits with spiking neural networks

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Exploring Optimized Spiking Neural Network Architectures for Classification Tasks on Embedded Platforms

Digit Recognition using Multimodal Spiking Neural Networks

Evaluation of Spiking Neural Nets-Based Image Classification Using the Runtime Simulator RAVSim

ACE-SNN: Algorithm-Hardware Co-design of Energy-Efficient & Low-Latency Deep Spiking Neural Networks for 3D Image Recognition

An FPGA Implementation of Deep Spiking Neural Networks for Low-Power and Fast Classification

Scalable Handwritten Digit Recognition Application using Neural Network and Convolutional Neural Network On Heterogeneous Architecture

Using CSNNs to Perform Event-based Data Processing & Classification on ASL-DVS

Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation

A Fast Spiking Neural Network Accelerator based on BP-STDP Algorithm and Weighted Neuron Model

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

Unsupervised learning of digit recognition using spike-timing-dependent plasticity

NeuroWrite: Predictive Handwritten Digit Classification using Deep Neural Networks

A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware

Image Edge Detector with Gabor Type Filters Using a Spiking Neural Network of Biologically Inspired Neurons

Direct training high-performance spiking neural networks for object recognition and detection

Direct Training for Spiking Neural Networks: Faster, Larger, Better

Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentation

Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks