Abstract:Spiking Neural Networks (SNNs) have indeed shown remarkable promise in the field of computer vision, emerging as a low-energy alternative to traditional Artificial Neural Networks (ANNs). However, SNNs also face several challenges: i) Existing SNNs are not purely additive and involve a substantial amount of floating-point computations, which contradicts the original design intention of adapting to neuromorphic chips; ii) The incorrect positioning of convolutional and pooling layers relative to spiking layers leads to reduced accuracy; iii) Leaky Integrate-and-Fire (LIF) neurons have limited capability in representing local information, which is disadvantageous for downstream visual tasks like semantic segmentation. To address the challenges in SNNs, i) we introduce Pure Sparse Self Attention (PSSA) and Dynamic Spiking Membrane Shortcut (DSMS), combining them to tackle the issue of floating-point computations; ii) the Spiking Precise Gradient downsampling (SPG-down) method is proposed for accurate gradient transmission; iii) the Group-LIF neuron concept is introduced to ensure LIF neurons' capability in representing local information both horizontally and vertically, enhancing their applicability in semantic segmentation tasks. Ultimately, these three solutions are integrated into the Powerful Sparse-Spike-Driven Transformer (PSSD-Transformer), effectively handling semantic segmentation tasks and addressing the challenges inherent in SNNs. The experimental results demonstrate that our model outperforms previous results on standard classification datasets and also shows commendable performance on semantic segmentation datasets. Up to this point, PSSD is the first model in the SNN field to perform semantic segmentation on large datasets. The code will be made publicly available after the paper is accepted for publication.

SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images

PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding

Few-Shot Panoptic Segmentation With Foundation Models

Hierarchical Open-vocabulary Universal Image Segmentation

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

Open-set Hierarchical Semantic Segmentation for 3D Scene

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

GSPN : Generative Shape Proposal Network for 3 D Instance Segmentation in Point Cloud – Supplementary Material

SPIN: Spacecraft Imagery for Navigation

Deep Hierarchical Semantic Segmentation

PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation

Image superpixel segmentation based on hierarchical multi-level LI-SLIC

SPIN Road Mapper: Extracting Roads from Aerial Images via Spatial and Interaction Space Graph Reasoning for Autonomous Driving

From Pixels to Objects: A Hierarchical Approach for Part and Object Segmentation Using Local and Global Aggregation

Semantic Segmentation for Urban-Scene Images

Search3D: Hierarchical Open-Vocabulary 3D Segmentation

Part2Object: Hierarchical Unsupervised 3D Instance Segmentation

View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields

Learning Part Segmentation from Synthetic Animals

3D Segmentation Learning from Sparse Annotations and Hierarchical Descriptors

Hierarchical Semantic Segmentation using Psychometric Learning