Abstract:As the representatives of brain-inspired models at the neuronal level, spiking neural networks (SNNs) have shown great promise in processing spatiotemporal information with intrinsic temporal dynamics. SNNs are expected to further improve their robustness and computing efficiency by introducing top-down attention at the architectural level, which is crucial for the human brain to support advanced intelligence. However, this attempt encounters difficulties in optimizing the attention in SNNs largely due to the lack of annotations. Here, we develop a hybrid network model with a top-down attention mechanism (HTDA) by incorporating an artificial neural network (ANN) to generate attention maps based on the features extracted by a feedforward SNN. The attention map is then used to modulate the encoding layer of the SNN so that it focuses on the most informative sensory input. To facilitate direct learning of attention maps and avoid labor-intensive annotations, we propose a general principle and a corresponding weakly-supervised objective, which promotes the HTDA model to utilize an integral and small subset of the input to give accurate predictions. On this basis, the ANN and the SNN can be jointly optimized by surrogate gradient descent in an end-to-end manner. We comprehensively evaluated the HTDA model on object recognition tasks, which demonstrates strong robustness to adversarial noise, high computing efficiency, and good interpretability. On the widely-adopted CIFAR-10, CIFAR-100, and MNIST benchmarks, the HTDA model reduces firing rates by up to 50% and improves adversarial robustness by up to 10% with comparable or better accuracy compared with the state-of-the-art SNNs. The HTDA model is also verified on dynamic neuromorphic datasets and achieves consistent improvements. This study provides a new way to boost the performance of SNNs by employing a hybrid top-down attention mechanism.

A Spatial–Channel–Temporal-Fused Attention for Spiking Neural Networks

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

Spatial-Temporal Self-Attention for Asynchronous Spiking Neural Networks

TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Event-Based Multimodal Spiking Neural Network with Attention Mechanism

Attention Spiking Neural Networks

STCA-SNN: Self-Attention-based Temporal-Channel Joint Attention for Spiking Neural Networks

Enhancing spiking neural networks with hybrid top-down attention

STSC-SNN: Spatio-Temporal Synaptic Connection with temporal convolution and attention for spiking neural networks

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Enhancing SNN-based Spatio-Temporal Learning: A Benchmark Dataset and Cross-Modality Attention Model

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Advancing Spiking Neural Networks towards Multiscale Spatiotemporal Interaction Learning

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Temporal-wise Attention Spiking Neural Networks for Event Streams Classification

Spiking Transformer with Spatial-Temporal Attention

Spatial-Temporal Search for Spiking Neural Networks

When Spiking neural networks meet temporal attention image decoding and adaptive spiking neuron

Event-based Action Recognition Using Motion Information and Spiking Neural Networks