Abstract:Fusing Events and RGB images for object detection leverages the robustness of Event cameras in adverse environments and the rich semantic information provided by RGB cameras. However, two critical mismatches: low-latency Events \textit{vs.}~high-latency RGB frames; temporally sparse labels in training \textit{vs.}~continuous flow in inference, significantly hinder the high-frequency fusion-based object detection. To address these challenges, we propose the \textbf{F}requency-\textbf{A}daptive Low-Latency \textbf{O}bject \textbf{D}etector (FAOD). FAOD aligns low-frequency RGB frames with high-frequency Events through an Align Module, which reinforces cross-modal style and spatial proximity to address the Event-RGB Mismatch. We further propose a training strategy, Time Shift, which enforces the module to align the prediction from temporally shifted Event-RGB pairs and their original representation, that is, consistent with Event-aligned annotations. This strategy enables the network to use high-frequency Event data as the primary reference while treating low-frequency RGB images as supplementary information, retaining the low-latency nature of the Event stream toward high-frequency detection. Furthermore, we observe that these corrected Event-RGB pairs demonstrate better generalization from low training frequency to higher inference frequencies compared to using Event data alone. Extensive experiments on the PKU-DAVIS-SOD and DSEC-Detection datasets demonstrate that our FAOD achieves SOTA performance. Specifically, in the PKU-DAVIS-SOD Dataset, FAOD achieves 9.8 points improvement in terms of the mAP in fully paired Event-RGB data with only a quarter of the parameters compared to SODFormer, and even maintains robust performance (only a 3 points drop in mAP) under 80$\times$ Event-RGB frequency mismatch.

Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion

Nighttime Traffic Object Detection Via Adaptively Integrating Event and Frame Domains

Global-local Feature Aggregation for Event-based Object Detection on EventKITTI

Embracing Events and Frames with Hierarchical Feature Refinement Network for Object Detection

Lightweight Real-Time Object Detection via Enhanced Global Perception and Intra-Layer Interaction for Complex Traffic Scenarios

An attention fusion network for event-based vehicle object detection

Traffic Sign Detection using Feature Fusion and Contextual Information

From Dense to Sparse: Low-Latency and Speed-Robust Event-Based Object Detection

Toward Effective Traffic Sign Detection via Two-Stage Fusion Neural Networks

Small Object Detection in Traffic Scenes Based on Attention Feature Fusion

FSFNet: Foreground Score-Aware Fusion for 3-D Object Detector under Unfavorable Conditions

Traffic Light Detection by Integrating Feature Fusion and Attention Mechanism

Traffic Lights Detection and Recognition Algorithm Based on Multi-feature Fusion

E-detector: Asynchronous Spatio-temporal for Event-based Object Detection in Intelligent Transportation System

A Fast RetinaNet Fusion Framework for Multi-Spectral Pedestrian Detection

Frequency-Adaptive Low-Latency Object Detection Using Events and Frames

TUMTraf Event: Calibration and Fusion Resulting in a Dataset for Roadside Event-Based and RGB Cameras

Spatio-temporal Focus and Lightweight Memory Network for Continuous Object Detection with Event Camera

Automatic Object Classification Using Motion Blob Based Local Feature Fusion for Traffic Scene Surveillance

Let There be Light: Improved Traffic Surveillance via Detail Preserving Night-to-Day Transfer

Motion Robust High-Speed Light-Weighted Object Detection With Event Camera