Abstract:Spiking Neural Networks (SNNs) have indeed shown remarkable promise in the field of computer vision, emerging as a low-energy alternative to traditional Artificial Neural Networks (ANNs). However, SNNs also face several challenges: i) Existing SNNs are not purely additive and involve a substantial amount of floating-point computations, which contradicts the original design intention of adapting to neuromorphic chips; ii) The incorrect positioning of convolutional and pooling layers relative to spiking layers leads to reduced accuracy; iii) Leaky Integrate-and-Fire (LIF) neurons have limited capability in representing local information, which is disadvantageous for downstream visual tasks like semantic segmentation. To address the challenges in SNNs, i) we introduce Pure Sparse Self Attention (PSSA) and Dynamic Spiking Membrane Shortcut (DSMS), combining them to tackle the issue of floating-point computations; ii) the Spiking Precise Gradient downsampling (SPG-down) method is proposed for accurate gradient transmission; iii) the Group-LIF neuron concept is introduced to ensure LIF neurons' capability in representing local information both horizontally and vertically, enhancing their applicability in semantic segmentation tasks. Ultimately, these three solutions are integrated into the Powerful Sparse-Spike-Driven Transformer (PSSD-Transformer), effectively handling semantic segmentation tasks and addressing the challenges inherent in SNNs. The experimental results demonstrate that our model outperforms previous results on standard classification datasets and also shows commendable performance on semantic segmentation datasets. Up to this point, PSSD is the first model in the SNN field to perform semantic segmentation on large datasets. The code will be made publicly available after the paper is accepted for publication.

Convolutional point transformer for semantic segmentation of sewer sonar point clouds

Sewer defect instance segmentation, localization, and 3D reconstruction for sewer floating capsule robots

PointMS: Semantic Segmentation for Point Cloud Based on Multi-scale Directional Convolution

PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

pCTFusion: Point Convolution-Transformer Fusion with Semantic Aware Loss for Outdoor LiDAR Point Cloud Segmentation

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

Sewer defect detection from 3D point clouds using a transformer-based deep learning model

Automatic sewer defect detection and severity quantification based on pixel-level semantic segmentation

MPCT: Multiscale Point Cloud Transformer with a Residual Network

ESP-PCT: Enhanced VR Semantic Performance through Efficient Compression of Temporal and Spatial Redundancies in Point Cloud Transformers

CNN and Transformer Fusion for Remote Sensing Image Semantic Segmentation

D2T-Net: A dual-domain transformer network exploiting spatial and channel dimensions for semantic segmentation of urban mobile laser scanning point clouds

An efficient point cloud semantic segmentation network with multiscale super-patch transformer

SPT-UNet: A Superpixel-Level Feature Fusion Network for Water Extraction from SAR Imagery

TransUPR: A Transformer-based Plug-and-Play Uncertain Point Refiner For LiDAR Point Cloud Semantic Segmentation

PointCAT: Cross-Attention Transformer for point cloud

PCT: Point cloud transformer

PVT: Point-Voxel Transformer for Point Cloud Learning

TSCB-Net: transformer-enhanced semantic segmentation of surface damage of concrete bridges

Incorporating convolutional and transformer architectures to enhance semantic segmentation of fine-resolution urban images