Abstract:Introduction: The spiking neural network (SNN) is a bionic model that is energy-efficient when implemented on neuromorphic hardwares. The non-differentiability of the spiking signals and the complicated neural dynamics make direct training of high-performance SNNs a great challenge. There are numerous crucial issues to explore for the deployment of direct training SNNs, such as gradient vanishing and explosion, spiking signal decoding, and applications in upstream tasks. Methods: To address gradient vanishing, we introduce a binary selection gate into the basic residual block and propose spiking gate (SG) ResNet to implement residual learning in SNNs. We propose two appropriate representations of the gate signal and verify that SG ResNet can overcome gradient vanishing or explosion by analyzing the gradient backpropagation. For the spiking signal decoding, a better decoding scheme than rate coding is achieved by our attention spike decoder (ASD), which dynamically assigns weights to spiking signals along the temporal, channel, and spatial dimensions. Results and discussion: The SG ResNet and ASD modules are evaluated on multiple object recognition datasets, including the static ImageNet, CIFAR-100, CIFAR-10, and neuromorphic DVS-CIFAR10 datasets. Superior accuracy is demonstrated with a tiny simulation time step of four, specifically 94.52% top-1 accuracy on CIFAR-10 and 75.64% top-1 accuracy on CIFAR-100. Spiking RetinaNet is proposed using SG ResNet as the backbone and ASD module for information decoding as the first direct-training hybrid SNN-ANN detector for RGB images. Spiking RetinaNet with a SG ResNet34 backbone achieves an mAP of 0.296 on the object detection dataset MSCOCO.

Direct training high-performance spiking neural networks for object recognition and detection

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Spiking Deep Residual Networks.

Direct Training for Spiking Neural Networks: Faster, Larger, Better

Spiking VGG7: Deep Convolutional Spiking Neural Network with Direct Training for Object Recognition

Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection

A Rapid and Precise Spiking Neural Network for Image Recognition

Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition

Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentation

R-SNN: Region-Based Spiking Neural Network for Object Detection

Deep Directly-Trained Spiking Neural Networks for Object Detection

Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance.

A Parallel Spiking Neural Network Based on Adaptive Lateral Inhibition Mechanism for Objective Recognition

Spiking Neural Networks for Object Detection Based on Integrating Neuronal Variants and Self-Attention Mechanisms

Enabling energy-Efficient object detection with surrogate gradient descent in spiking neural networks

Spiking Neural Network for Ultra-low-latency and High-accurate Object Detection

Object Detection Method with Spiking Neural Network Based on DT-LIF Neuron and SSD

Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection

Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detection