Abstract:Although spiking neural networks (SNNs) have made great progress on both performance and efficiency over the last few years, their unique working pattern makes it hard to train high-performance low-latency SNNs and their development still lags behind traditional artificial neural networks (ANNs). To compensate this gap, many extraordinary works have been proposed, but these works are mainly based on the same network structure (i.e. CNN) and their performance is worse than their ANN counterparts, which limits the applications of SNNs. To this end, we propose a Transformer-based SNN, termed ”Spikeformer”, which outperforms its ANN counterpart on both static dataset and neuromorphic datasets. First, to deal with the problem of “data hungry” and the unstable training period exhibited in the vanilla model, we design the Convolutional Tokenizer (CT) module, which stabilizes training and improves the accuracy of the original model on DVS-Gesture by more than 16%. Besides, we integrate Spatio-Temporal Attention (STA) into Spikeformer to better incorporate the attention mechanism inside Transformer and the spatio-temporal information inherent to SNN. With our proposed method, we achieve 98.96%/75.89% top-1 accuracy on DVS-Gesture/ImageNet datasets with 16/4 simulation time steps. On DVS-CIFAR10, we further conduct energy consumption analysis and obtain 81.4%/80.3% top-1 accuracy with 4/1 time step(s), achieving 1.7/6.4 × energy efficiency over its ANN counterpart. Moreover, our Spikeformer outperforms its ANN counterpart by 3.13% and 0.12% on DVS-Gesture and ImageNet respectively, indicating that Spikeformer may be a more suitable architecture for training SNNs compared to CNN. We believe that this work shall promote the development of SNNs to be in step with ANNs as much as possible. Code will be publicly available.

SpikeFusionNet: A Hybrid Approach to Robotic Fault Diagnosis Using Spiking Neural Dynamics

An event-driven Spike-DBN model for fault diagnosis using reward-STDP

Online Fault Diagnosis Using Bioinspired Spike Neural Network

BayesianSpikeFusion: accelerating spiking neural network inference via Bayesian fusion of early prediction

Spike Trains Encoding Optimization for Spiking Neural Networks Implementation in FPGA

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

A Novel Label-Aware Global Graph Construction Method and Spiking-Coded Graph Neural Network for Intelligent Process Fault Diagnosis

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Toward End-to-End Bearing Fault Diagnosis for Industrial Scenarios with Spiking Neural Networks

SpikeFI: A Fault Injection Framework for Spiking Neural Networks

The Lightweight Edge-Side Fault Diagnosis Approach Based on Spiking Neural Network

Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training

Towards Scalable GPU-Accelerated SNN Training via Temporal Fusion

MT-Spike: A Multilayer Time-based Spiking Neuromorphic Architecture with Temporal Error Backpropagation

Improved spiking neural network for intershaft bearing fault diagnosis

A Parallel Spiking Neural Network Based on Adaptive Lateral Inhibition Mechanism for Objective Recognition

A Bayesian CNN-based Fusion Framework of Sensor Fault Diagnosis

Spikeformer: Training high-performance spiking neural network with transformer

Training Spiking Neural Networks for Cognitive Tasks: A Versatile Framework Compatible with Various Temporal Codes.

Feature fusion method based on spiking neural convolutional network for edge detection