Abstract:Various hypotheses of information representation in brain, referred to as neural codes, have been proposed to explain the information transmission between neurons. Neural coding plays an essential role in enabling the brain-inspired spiking neural networks (SNNs) to perform different tasks. To search for the best coding scheme, we performed an extensive comparative study on the impact and performance of four important neural coding schemes, namely, rate coding, time-to-first spike (TTFS) coding, phase coding, and burst coding. The comparative study was carried out using a biological 2-layer SNN trained with an unsupervised spike-timing-dependent plasticity (STDP) algorithm. Various aspects of network performance were considered, including classification accuracy, processing latency, synaptic operations (SOPs), hardware implementation, network compression efficacy, input and synaptic noise resilience, and synaptic fault tolerance. The classification tasks on Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets were applied in our study. For hardware implementation, area and power consumption were estimated for these coding schemes, and the network compression efficacy was analyzed using pruning and quantization techniques. Different types of input noise and noise variations in the datasets were considered and applied. Furthermore, the robustness of each coding scheme to the non-ideality-induced synaptic noise and fault in analog neuromorphic systems was studied and compared. Our results show that TTFS coding is the best choice in achieving the highest computational performance with very low hardware implementation overhead. TTFS coding requires 4x/7.5x lower processing latency and 3.5x/6.5x fewer SOPs than rate coding during the training/inference process. Phase coding is the most resilient scheme to input noise. Burst coding offers the highest network compression efficacy and the best overall robustness to hardware non-idealities for both training and inference processes. The study presented in this paper reveals the design space created by the choice of each coding scheme, allowing designers to frame each scheme in terms of its strength and weakness given a designs’ constraints and considerations in neuromorphic systems.

A spiking photonic neural network of 40.000 neurons, trained with rank-order coding for leveraging sparsity

Exploiting Noise As a Resource for Computation and Learning in Spiking Neural Networks

High-performance deep spiking neural networks with 0.3 spikes per neuron

Photonic Spiking Neural Networks with Highly Efficient Training Protocols for Ultrafast Neuromorphic Computing Systems

A Comparative Study on the Performance and Security Evaluation of Spiking Neural Networks

Neural Coding in Spiking Neural Networks: A Comparative Study for Robust Neuromorphic Systems

A 4096-Neuron 1M-Synapse 3.8-pJ/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS

Optimized spiking neurons classify images with high accuracy through temporal coding with two spikes

A Digital Neuromorphic Hardware for Spiking Neural Network

Improving classification accuracy of feedforward neural networks for spiking neuromorphic chips

Deep Spiking Convolutional Neural Networks for Programmable Neuro-synaptic System

NBSSN: A Neuromorphic Binary Single-Spike Neural Network for Efficient Edge Intelligence.

Algorithm and hardware design of discrete-time spiking neural networks based on back propagation with binary activations

A 0.67-to-5.4 TSOPs/W Spiking Neural Network Accelerator With 128/256 Reconfigurable Neurons and Asynchronous Fully Connected Synapses

Turn Down that Noise: Synaptic Encoding of Afferent SNR in a Single Spiking Neuron

Photonic spiking neural networks with event-driven femtojoule optoelectronic neurons based on Izhikevich-inspired model

A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation

On-chip Spiking Neural Networks Based on Add-Drop Ring Microresonators and Electrically Reconfigurable Phase-Change Material Photonic Switches

A A 22nm 0.43pJ/SOP Sparsity-Aware In-Memory Neuromorphic Computing System with Hybrid Spiking and Artificial Neural Network and Configurable Topology

A Large-Scale Photonic CNN Based on Spike Coding and Temporal Integration