Abstract:The ability to detect objects in all lighting (i.e., normal-, over-, and under-exposed) conditions is crucial for real-world applications, such as <a class="link-external link-http" href="http://self-driving.Traditional" rel="external noopener nofollow">this http URL</a> RGB-based detectors often fail under such varying lighting <a class="link-external link-http" href="http://conditions.Therefore" rel="external noopener nofollow">this http URL</a>, recent works utilize novel event cameras to supplement or guide the RGB modality; however, these methods typically adopt asymmetric network structures that rely predominantly on the RGB modality, resulting in limited robustness for all-day detection. In this paper, we propose EOLO, a novel object detection framework that achieves robust and efficient all-day detection by fusing both RGB and event modalities. Our EOLO framework is built based on a lightweight spiking neural network (SNN) to efficiently leverage the asynchronous property of events. Buttressed by it, we first introduce an Event Temporal Attention (ETA) module to learn the high temporal information from events while preserving crucial edge information. Secondly, as different modalities exhibit varying levels of importance under diverse lighting conditions, we propose a novel Symmetric RGB-Event Fusion (SREF) module to effectively fuse RGB-Event features without relying on a specific modality, thus ensuring a balanced and adaptive fusion for all-day detection. In addition, to compensate for the lack of paired RGB-Event datasets for all-day training and evaluation, we propose an event synthesis approach based on the randomized optical flow that allows for directly generating the event frame from a single exposure image. We further build two new datasets, E-MSCOCO and E-VOC based on the popular benchmarks MSCOCO and PASCAL VOC. Extensive experiments demonstrate that our EOLO outperforms the state-of-the-art detectors,e.g.,RENet,by a substantial margin (+3.74% mAP50) in all lighting <a class="link-external link-http" href="http://conditions.Our" rel="external noopener nofollow">this http URL</a> code and datasets will be available at <a class="link-external link-https" href="https://vlislab22.github.io/EOLO/" rel="external noopener nofollow">this https URL</a>

Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera

A Lightweight SE-YOLOv3 Network for Multi-Scale Object Detection in Remote Sensing Imagery.

A Recurrent YOLOv8-based framework for Event-Based Object Detection

Detecting Every Object from Events

Seeing Motion at Nighttime with an Event Camera

Backlight and dim space object detection based on a novel event camera

Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Dataset

Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion

Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric

Better and Faster: Adaptive Event Conversion for Event-Based Object Detection.

LEDet: A Single-Shot Real-Time Object Detector Based on Low-Light Image Enhancement

Motion Robust High-Speed Light-Weighted Object Detection With Event Camera

Frequency-Adaptive Low-Latency Object Detection Using Events and Frames

TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking

A novel low light object detection method based on the YOLOv5 fusion feature enhancement

YOLO-DA: An Efficient YOLO-Based Detector for Remote Sensing Object Detection

EMNet: Edge-guided multi-level network for salient object detection in low-light images

Event-assisted Low-Light Video Object Segmentation

Event-Based Low-Illumination Image Enhancement

YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features

Real-Time One-Stream Semantic-Guided Refinement Network for RGB-Thermal Salient Object Detection