Abstract:Event cameras, with their high dynamic range and temporal resolution, are ideally suited for object detection, especially under scenarios with motion blur and challenging lighting conditions. However, while most existing approaches prioritize optimizing spatiotemporal representations with advanced detection backbones and early aggregation functions, the crucial issue of adaptive event sampling remains largely unaddressed. Spiking Neural Networks (SNNs), which operate on an event-driven paradigm through sparse spike communication, emerge as a natural fit for addressing this challenge. In this study, we discover that the neural dynamics of spiking neurons align closely with the behavior of an ideal temporal event sampler. Motivated by this insight, we propose a novel adaptive sampling module that leverages recurrent convolutional SNNs enhanced with temporal memory, facilitating a fully end-to-end learnable framework for event-based detection. Additionally, we introduce Residual Potential Dropout (RPD) and Spike-Aware Training (SAT) to regulate potential distribution and address performance degradation encountered in spike-based sampling modules. Empirical evaluation on neuromorphic detection datasets demonstrates that our approach outperforms existing state-of-the-art spike-based methods with significantly fewer parameters and time steps. For instance, our method yields a 4.4\% mAP improvement on the Gen1 dataset, while requiring 38\% fewer parameters and only three time steps. Moreover, the applicability and effectiveness of our adaptive sampling methodology extend beyond SNNs, as demonstrated through further validation on conventional non-spiking models. Code is available at <a class="link-external link-https" href="https://github.com/Windere/EAS-SNN" rel="external noopener nofollow">this https URL</a>.

Towards event camera signal recognition with lightweight spiking

Towards Event Camera Signal Recognition Using a Lightweight Spiking Neural Network

SpikePoint: An Efficient Point-based Spiking Neural Network for Event Cameras Action Recognition

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Event-based Action Recognition Using Motion Information and Spiking Neural Networks

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

[A bio-inspired hierarchical spiking neural network with biological synaptic plasticity for event camera object recognition].

Spiking Neural Network Recognition Method Based on Dynamic Visual Motion Features

An Event-based Feature Representation Method for Event Stream Classification Using Deep Spiking Neural Networks

Spiking Neural Network as Adaptive Event Stream Slicer

An Event-Driven Computational System With Spiking Neurons For Object Recognition

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks

Lightweight Convolutional SNN for Address Event Representation Signal Recognition.

Event-driven Spiking Neural Networks with Spike-Based Learning

Eye Tracking Based on Event Camera and Spiking Neural Network

EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networks

Spiking-Fer: Spiking Neural Network for Facial Expression Recognition With Event Cameras

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Event-based Object Detection with Lightweight Spatial Attention Mechanism