Abstract:Traffic object detection under variable illumination is challenging due to the information loss caused by the limited dynamic range of conventional frame-based cameras. To address this issue, we introduce bio-inspired event cameras and propose a novel Structure-aware Fusion Network (SFNet) that extracts sharp and complete object structures from the event stream to compensate for the lost information in images through cross-modality fusion, enabling the network to obtain illumination-robust representations for traffic object detection. Specifically, to mitigate the sparsity or blurriness issues arising from diverse motion states of traffic objects in fixed-interval event sampling methods, we propose the Reliable Structure Generation Network (RSGNet) to generate Speed Invariant Frames (SIF), ensuring the integrity and sharpness of object structures. Next, we design a novel Adaptive Feature Complement Module (AFCM) which guides the adaptive fusion of two modality features to compensate for the information loss in the images by perceiving the global lightness distribution of the images, thereby generating illumination-robust representations. Finally, considering the lack of large-scale and high-quality annotations in the existing event-based object detection datasets, we build a DSEC-Det dataset, which consists of 53 sequences with 63,931 images and more than 208,000 labels for 8 classes. Extensive experimental results demonstrate that our proposed SFNet can overcome the perceptual boundaries of conventional cameras and outperform the frame-based method by 8.0% in mAP50 and 5.9% in mAP50:95. Our code and dataset will be available at <a class="link-external link-https" href="https://github.com/YN-Yang/SFNet" rel="external noopener nofollow">this https URL</a>.

Local Fast R-Cnn Flow For Object-Centric Event Recognition In Complex Traffic Scenes

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

Spatiotemporal Analysis of Static and Dynamic Traffic Elements from Road Scenes.

Integrated Multi-Scale Event Verification In An Augmented Foreground Motion Space

A method of cross-layer fusion multi-object detection and recognition based on improved faster R-CNN model in complex traffic environment

Road Traffic Sign Detection Method Based on RTS R-CNN Instance Segmentation Network

Traffic Event Representation and Recognition Based on Context

Application Research of Key Frames Extraction Technology Combined with Optimized Faster R-CNN Algorithm in Traffic Video Analysis

Lightweight Real-Time Object Detection via Enhanced Global Perception and Intra-Layer Interaction for Complex Traffic Scenarios

Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation

Traffic scene recognition based on deep cnn and vlad spatial pyramids

LCIF-Net: Local criss-cross attention based optical flow method using multi-scale image features and feature pyramid

Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion

FPCR-Net: Feature pyramidal correlation and residual reconstruction for optical flow estimation

Two-Stage Adaptive Object Scene Flow Using Hybrid CNN-CRF Model

HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embedding

3D object recognition method with multiple feature extraction from LiDAR point clouds

Smart traffic management of vehicles using faster R-CNN based deep learning method

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

Simultaneous detection and tracking using deep learning and integrated channel feature for ambint traffic light recognition