Abstract:Spiking Neural Networks (SNNs) have indeed shown remarkable promise in the field of computer vision, emerging as a low-energy alternative to traditional Artificial Neural Networks (ANNs). However, SNNs also face several challenges: i) Existing SNNs are not purely additive and involve a substantial amount of floating-point computations, which contradicts the original design intention of adapting to neuromorphic chips; ii) The incorrect positioning of convolutional and pooling layers relative to spiking layers leads to reduced accuracy; iii) Leaky Integrate-and-Fire (LIF) neurons have limited capability in representing local information, which is disadvantageous for downstream visual tasks like semantic segmentation. To address the challenges in SNNs, i) we introduce Pure Sparse Self Attention (PSSA) and Dynamic Spiking Membrane Shortcut (DSMS), combining them to tackle the issue of floating-point computations; ii) the Spiking Precise Gradient downsampling (SPG-down) method is proposed for accurate gradient transmission; iii) the Group-LIF neuron concept is introduced to ensure LIF neurons' capability in representing local information both horizontally and vertically, enhancing their applicability in semantic segmentation tasks. Ultimately, these three solutions are integrated into the Powerful Sparse-Spike-Driven Transformer (PSSD-Transformer), effectively handling semantic segmentation tasks and addressing the challenges inherent in SNNs. The experimental results demonstrate that our model outperforms previous results on standard classification datasets and also shows commendable performance on semantic segmentation datasets. Up to this point, PSSD is the first model in the SNN field to perform semantic segmentation on large datasets. The code will be made publicly available after the paper is accepted for publication.

Semantic Interpretation for Convolutional Neural Networks: What Makes a Cat a Cat?

AS-XAI: Self-supervised Automatic Semantic Interpretation for CNN

A Pixel-Level Explainable Approach of Convolutional Neural Networks and Its Application

Automated Natural Language Explanation of Deep Visual Neurons with Large Models

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Improving Interpretability of Deep Neural Networks with Semantic Information

Deeper Interpretability of Deep Networks

PSSD-Transformer: Powerful Sparse Spike-Driven Transformer for Image Semantic Segmentation

Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?

Explainable AI: A review of applications to neuroimaging data

Spatial-temporal Concept Based Explanation of 3D ConvNets.

Embedding deep networks into visual explanations

Beyond the Veil of Similarity: Quantifying Semantic Continuity in Explainable AI

Deep Semantic Understanding of High Resolution Remote Sensing Image

Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations

Explaining Deep Convolutional Neural Networks via Latent Visual-Semantic Filter Attention

Semantic Neural Decoding via Cross-Modal Generation

Explain Any Concept: Segment Anything Meets Concept-Based Explanation

Feature-Based Interpretation of Image Classification With the Use of Convolutional Neural Networks

A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts

Hybrid CNN -Interpreter: Interpret local and global contexts for CNN-based Models