Abstract:Spiking neural networks (SNNs), as the brain-inspired neural networks, encode information in spatio-temporal dynamics. They have the potential to serve as low-power alternatives to artificial neural networks (ANNs) due to their sparse and event-driven nature. However, existing SNN-based models for pixel-level semantic segmentation tasks suffer from poor performance and high memory overhead, failing to fully exploit the computational effectiveness and efficiency of SNNs. To address these challenges, we propose the multi-scale and full spike segmentation network (MFS-Seg), which is based on the deep direct trained SNN and represents the first attempt to train a deep SNN with surrogate gradients for semantic segmentation. Specifically, we design an efficient fully-spike residual block (EFS-Res) to alleviate representation issues caused by spiking noise on different channels. EFS-Res utilizes depthwise separable convolution to improve the distributions of spiking feature maps. The visualization shows that our model can effectively extract the edge features of segmented objects. Furthermore, it can significantly reduce the memory overhead and energy consumption of the network. In addition, we theoretically analyze and prove that EFS-Res can avoid the degradation problem based on block dynamical isometry theory. Experimental results on the Camvid dataset, the DDD17 dataset, and the DSEC-Semantic dataset show that our model achieves comparable performance to the mainstream UNet network with up to 31× fewer parameters, while significantly reducing power consumption by over 13×. Overall, our MFS-Seg model demonstrates promising results in terms of performance, memory efficiency, and energy consumption, showcasing the potential of deep SNNs for semantic segmentation tasks. Our code is available in https://github.com/BICLab/MFS-Seg.

A Dynamic Region Generation Algorithm For Image Segmentation Based On Spiking Neural Network

Image Semantic Segmentation Based on Region and Deep Residual Network

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

A Parallel Convolutional Network Based on Spiking Neural Systems

Automatic Image Segmentation by Dynamic Region Merging

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

An Image Segmentation Approach Based on Fuzzy-Neural-network Hybrid System

Spiking neural networks fine-tuning for brain image segmentation

Multi-scale Full Spike Pattern for Semantic Segmentation

A Spiking Neural Network for Image Segmentation

Dynamic Spatiotemporal Pattern Recognition with Recurrent Spiking Neural Network

A 67.5μJ/Prediction Accelerator for Spiking Neural Networks in Image Segmentation

Automated scalable segmentation of neurons from multispectral images

Image Segmentation Based On Pulse Coupled Neural Network

Energy-Efficient Spiking Segmenter for Frame and Event-Based Images.

Improved DCNON Segmentation Algorithm Research in Color Image Sphere Coordinate

MSDEnet: Multi-scale detail enhanced network based on human visual system for medical image segmentation

SegNet Network Architecture for Deep Learning Image Segmentation and Its Integrated Applications and Prospects

Keep the geometries: Image segmentation by K-MSVC with random region grouping and propagation

A Novel Approach to Optimizing Convolutional Neural Networks for Improved Digital Image Segmentation

Rethinking Superpixel Segmentation from Biologically Inspired Mechanisms