Abstract:The event camera, as an innovative perception sensor, generates a high-temporal-resolution event stream, filtering out redundant visual information and offering a wide dynamic range for object detection. Nevertheless, it is non-trivial to effectively utilize the advantages of the event stream to enhance the accuracy of object detection. Initially, current event representation methods require hyperparameters tuning to achieve optimal performance for event-based object detection, which is very time-consuming. Moreover, most existing methods rely on large-scale models to enhance detection accuracy. In addition, they fail to leverage the temporal clues provided by the event stream efficiently. In this work, we propose a detection pipeline which better achieves equilibrium between model speed and accuracy. First, we design an event representation without hyperparameter tuning named Dynamic Temporal Representation (DTR), which efficiently utilizes the spatial-temporal asynchronous event stream. Then, a module named Spatial Temporal Focus (STF), capable of extracting the rich temporal information from the DTR tensor, is proposed. Besides, a memory module is implemented, leveraging rich temporal clues named Light Recurrent Convolution (LRC). Finally, experiments demonstrate that our method outperforms state-of-the-art approaches on two classic event camera datasets. Specifically, in the Gen1 Automotive Dataset and the 1Mpx Detection Dataset, mean Average Precision (mAP) increases by 0.5% and 0.4%, respectively, while inference times decrease by 34.3% and 32.5%. Additionally, the parameters of the model are reduced by 50.1%. Therefore, our method demonstrates competitive advantages in terms of accuracy, efficiency, and parameters.

Spatio-Temporal Aggregation Transformer for Object Detection with Neuromorphic Vision Sensors

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection

SpikingViT: a Multi-scale Spiking Vision Transformer Model for Event-based Object Detection

An Event-Driven Object Recognition Model Using Activated Connected Domain Detection

Global-local Feature Aggregation for Event-based Object Detection on EventKITTI

Scene Adaptive Sparse Transformer for Event-based Object Detection

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Spiking Transformers for Event-based Single Object Tracking

Adaptive sparse attention-based compact transformer for object tracking

Asynchronous Spatio-Temporal Memory Network for Continuous Event-Based Object Detection

Adaptive Vision Transformer for Event-Based Human Pose Estimation

HDI-Former: Hybrid Dynamic Interaction ANN-SNN Transformer for Object Detection Using Frames and Events

Spatio-temporal Focus and Lightweight Memory Network for Continuous Object Detection with Event Camera

SODFormer: Streaming Object Detection with Transformer Using Events and Frames

Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection

GET: Group Event Transformer for Event-Based Vision

Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams

DS2TA: Denoising Spiking Transformer with Attenuated Spatiotemporal Attention

STAT: Spatial-Temporal Attention Mechanism for Video Captioning