Abstract:Spiking Neural Networks (SNNs) have indeed shown remarkable promise in the field of computer vision, emerging as a low-energy alternative to traditional Artificial Neural Networks (ANNs). However, SNNs also face several challenges: i) Existing SNNs are not purely additive and involve a substantial amount of floating-point computations, which contradicts the original design intention of adapting to neuromorphic chips; ii) The incorrect positioning of convolutional and pooling layers relative to spiking layers leads to reduced accuracy; iii) Leaky Integrate-and-Fire (LIF) neurons have limited capability in representing local information, which is disadvantageous for downstream visual tasks like semantic segmentation. To address the challenges in SNNs, i) we introduce Pure Sparse Self Attention (PSSA) and Dynamic Spiking Membrane Shortcut (DSMS), combining them to tackle the issue of floating-point computations; ii) the Spiking Precise Gradient downsampling (SPG-down) method is proposed for accurate gradient transmission; iii) the Group-LIF neuron concept is introduced to ensure LIF neurons' capability in representing local information both horizontally and vertically, enhancing their applicability in semantic segmentation tasks. Ultimately, these three solutions are integrated into the Powerful Sparse-Spike-Driven Transformer (PSSD-Transformer), effectively handling semantic segmentation tasks and addressing the challenges inherent in SNNs. The experimental results demonstrate that our model outperforms previous results on standard classification datasets and also shows commendable performance on semantic segmentation datasets. Up to this point, PSSD is the first model in the SNN field to perform semantic segmentation on large datasets. The code will be made publicly available after the paper is accepted for publication.

Boosting 3D Neuron Segmentation with 2D Vision Transformer Pre-trained on Natural Images

DINeuro: Distilling Knowledge from 2D Natural Images via Deformable Tubular Transferring Strategy for 3D Neuron Reconstruction

TF-Unet:An Automatic Cardiac MRI Image Segmentation Method

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

NRTR: Neuron Reconstruction with Transformer from 3D Optical Microscopy Images

Neuron Image Segmentation via Learning Deep Features and Enhancing Weak Neuronal Structures

Improved V-Net Based Image Segmentation for 3D Neuron Reconstruction.

Neuron segmentation using 3D wavelet integrated encoder-decoder network

UNetFormer: A Unified Vision Transformer Model and Pre-Training Framework for 3D Medical Image Segmentation

Adapting Pre-trained Vision Transformers from 2D to 3D through Weight Inflation Improves Medical Image Segmentation

Segmenting Neuronal Structure in 3D Optical Microscope Images Via Knowledge Distillation with Teacher-Student Network

TransSea: Hybrid CNN-Transformer with Semantic Awareness for 3D Brain Tumor Segmentation

Going Beyond U-Net: Assessing Vision Transformers for Semantic Segmentation in Microscopy Image Analysis

A neuron image segmentation method based Deep Boltzmann Machine and CV model.

3D Brainformer: 3D Fusion Transformer for Brain Tumor Segmentation

TransSea: Hybrid CNN–Transformer With Semantic Awareness for 3-D Brain Tumor Segmentation

SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentation

3D Neuron Microscopy Image Segmentation Via the Ray-Shooting Model and a DC-BLSTM Network.

UNETR: Transformers for 3D Medical Image Segmentation

Nnformer: Volumetric Medical Image Segmentation Via a 3D Transformer

3D TransUNet: Advancing Medical Image Segmentation through Vision Transformers