Abstract:Event-driven spiking neural networks (SNNs) have demonstrated significant potential for achieving high energy and area efficiency. However, existing SNN accelerators suffer from issues such as high latency and energy consumption due to serial accumulation-comparison operations. This is mainly because SNN neurons integrate spikes, accumulate membrane potential, and generate output spikes when the potential exceeds a threshold. To address this, one approach is to leverage the sparsity of SNN spikes to reduce the number of time steps. However, this method can result in imbalanced workloads among neurons and limit the utilization of processing elements (PEs). In this paper, we present SATO, a temporal-parallel SNN accelerator that enables parallel accumulation of membrane potential for all time steps. SATO adopts a two-stage pipeline methodology, effectively decoupling neuron computations. This not only maintains accuracy but also unveils opportunities for fine-grained parallelism. By dividing the neuron computation into distinct stages, SATO enables the concurrent execution of spike accumulation for each time step, leveraging the parallel processing capabilities of modern hardware architectures. This not only enhances the overall efficiency of the accelerator but also reduces latency by exploiting parallelism at a granular level. The architecture of SATO includes a novel binary adder-search tree for generating the output spike train, effectively decoupling the chronological dependence in the accumulation-comparison operation. Furthermore, SATO employs a bucket-sort-based method to evenly distribute compressed workloads to all PEs, maximizing data locality of input spike trains. Experimental results on various SNN models demonstrate that SATO outperforms the well-known accelerator, the 8-bit version of "Eyeriss" by 20.7× in terms of speedup and 6.0× energy-saving, on average. Compared to the state-of-the-art SNN accelerator "SpinalFlow", SATO can also achieve 4.6× performance gain and 3.1× energy reduction on average, which is quite impressive for inference.

A Structure–time Parallel Implementation of Spike-Based Deep Learning

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Spike Trains Encoding Optimization for Spiking Neural Networks Implementation in FPGA

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Spike Trains Encoding and Threshold Rescaling Method for Deep Spiking Neural Networks

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

MT-Spike: A Multilayer Time-based Spiking Neuromorphic Architecture with Temporal Error Backpropagation

Spiking Deep Residual Networks.

A Spike-Timing Based Integrated Model for Pattern Recognition

Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks

SPAT: FPGA-based Sparsity-Optimized Spiking Neural Network Training Accelerator with Temporal Parallel Dataflow

Comprehensive Online Training and Deployment for Spiking Neural Networks

High-parallelism Inception-like Spiking Neural Networks for Unsupervised Feature Learning

A Parallel Spiking Neural Network Based on Adaptive Lateral Inhibition Mechanism for Objective Recognition

Fast-Learning and High-Robustness Unsupervised Spiking Neural Networks with a Highly Parallel Inception-like Architecture

Spike Calibration: Fast and Accurate Conversion of Spiking Neural Network for Object Detection and Segmentation

PT-Spike: A Precise-Time-Dependent Single Spike Neuromorphic Architecture with Efficient Supervised Learning

Exploiting Temporal-Unrolled Parallelism for Energy-Efficient SNN Acceleration

Direct Training for Spiking Neural Networks: Faster, Larger, Better