Abstract:Introduction: The spiking neural network (SNN) is a bionic model that is energy-efficient when implemented on neuromorphic hardwares. The non-differentiability of the spiking signals and the complicated neural dynamics make direct training of high-performance SNNs a great challenge. There are numerous crucial issues to explore for the deployment of direct training SNNs, such as gradient vanishing and explosion, spiking signal decoding, and applications in upstream tasks. Methods: To address gradient vanishing, we introduce a binary selection gate into the basic residual block and propose spiking gate (SG) ResNet to implement residual learning in SNNs. We propose two appropriate representations of the gate signal and verify that SG ResNet can overcome gradient vanishing or explosion by analyzing the gradient backpropagation. For the spiking signal decoding, a better decoding scheme than rate coding is achieved by our attention spike decoder (ASD), which dynamically assigns weights to spiking signals along the temporal, channel, and spatial dimensions. Results and discussion: The SG ResNet and ASD modules are evaluated on multiple object recognition datasets, including the static ImageNet, CIFAR-100, CIFAR-10, and neuromorphic DVS-CIFAR10 datasets. Superior accuracy is demonstrated with a tiny simulation time step of four, specifically 94.52% top-1 accuracy on CIFAR-10 and 75.64% top-1 accuracy on CIFAR-100. Spiking RetinaNet is proposed using SG ResNet as the backbone and ASD module for information decoding as the first direct-training hybrid SNN-ANN detector for RGB images. Spiking RetinaNet with a SG ResNet34 backbone achieves an mAP of 0.296 on the object detection dataset MSCOCO.

Object detection and tracking method based on spiking neuron network

Spiking SiamFC++: Deep Spiking Neural Network for Object Tracking

Spike Trains Encoding and Threshold Rescaling Method for Deep Spiking Neural Networks

Direct training high-performance spiking neural networks for object recognition and detection

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Deep Spiking Neural Networks with Binary Weights for Object Recognition

Spike Calibration: Fast and Accurate Conversion of Spiking Neural Network for Object Detection and Segmentation

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Spiking VGG7: Deep Convolutional Spiking Neural Network with Direct Training for Object Recognition

Spiking Neural Network for Ultra-low-latency and High-accurate Object Detection

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Spiking Deep Residual Networks.

Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection

Low Latency of object detection for spikng neural network

A Spike Neural Network Model for Lateral Suppression of Spike-Timing-Dependent Plasticity with Adaptive Threshold

Spiking Transformers for Event-based Single Object Tracking

Spiking Neural Network for Ultralow-Latency and High-Accurate Object Detection

Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition

Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection

Object Tracking Algorithm Based on Channel-interconnection-spatial Attention Mechanism and Siamese Region Proposal Network

Toward High-Accuracy and Low-Latency Spiking Neural Networks With Two-Stage Optimization