Abstract:Spiking neural networks (SNNs), as the brain-inspired neural networks, encode information in spatio-temporal dynamics. They have the potential to serve as low-power alternatives to artificial neural networks (ANNs) due to their sparse and event-driven nature. However, existing SNN-based models for pixel-level semantic segmentation tasks suffer from poor performance and high memory overhead, failing to fully exploit the computational effectiveness and efficiency of SNNs. To address these challenges, we propose the multi-scale and full spike segmentation network (MFS-Seg), which is based on the deep direct trained SNN and represents the first attempt to train a deep SNN with surrogate gradients for semantic segmentation. Specifically, we design an efficient fully-spike residual block (EFS-Res) to alleviate representation issues caused by spiking noise on different channels. EFS-Res utilizes depthwise separable convolution to improve the distributions of spiking feature maps. The visualization shows that our model can effectively extract the edge features of segmented objects. Furthermore, it can significantly reduce the memory overhead and energy consumption of the network. In addition, we theoretically analyze and prove that EFS-Res can avoid the degradation problem based on block dynamical isometry theory. Experimental results on the Camvid dataset, the DDD17 dataset, and the DSEC-Semantic dataset show that our model achieves comparable performance to the mainstream UNet network with up to 31× fewer parameters, while significantly reducing power consumption by over 13×. Overall, our MFS-Seg model demonstrates promising results in terms of performance, memory efficiency, and energy consumption, showcasing the potential of deep SNNs for semantic segmentation tasks. Our code is available in https://github.com/BICLab/MFS-Seg.

Spiking Neural P System with Synaptic Vesicles and Applications in Multiple Brain Metastasis Segmentation

Hybrid Neural-like P Systems with Evolutionary Channels for Multiple Brain Metastases Segmentation

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

Spiking Neural P Systems With Microglia

A Novel Spiking Neural P System for Image Recognition.

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Multi-scale Full Spike Pattern for Semantic Segmentation

Spiking Neural P Systems with Multiple Channels and Anti-Spikes

Asynchronous Numerical Spiking Neural Membrane Systems with Local Synchronization

Spiking Neural P Systems with Delay on Synapses

Spiking neural networks fine-tuning for brain image segmentation

A Parallel Spiking Neural Network Based on Adaptive Lateral Inhibition Mechanism for Objective Recognition

Comprehensive Online Training and Deployment for Spiking Neural Networks

Spiking Neural P Systems with Cooperative Synapses

Spiking Neural P Systems with Mute Rules

Research on Compound Numerical Spiking Neural P System

Spike-Weighted Spiking Neural Network with Spiking Long Short-Term Memory: A Biomimetic Approach to Decoding Brain Signals

PMSN: A Parallel Multi-compartment Spiking Neuron for Multi-scale Temporal Processing

Spiking Neural Membrane Systems with Adaptive Synaptic Time Delay

Deep synergetic spiking neural P systems for the overall survival time prediction of glioblastoma patients