Abstract:Sensors have different perceptive abilities against environment. Sensor fusion plays a crucial role at achieving better perception by accumulating the information acquired at different times. But, the decision of the observation may conflict with each other due to usage of different algorithms, thresholds on processing algorithms, and different perceptive character of sensors. This study presents the late fusion method applied to outputs provided by deep learning models fed by camera and lidar sensor's measurement data. For camera sensor, a deep learning model as a multi-task network is proposed to multi-classify cars, motorcycles, bicycles, buses, trucks, and pedestrians under the category of dynamic traffic objects. In addition, color classified traffic lights and traffic signs with a capability of segmenting drivable area and detecting lane lines are classified under the category of static traffic objects. The proposed multi-network is trained and tested with the BDD100K dataset and benchmarked with publicly available multi-networks. The presented method is the second fastest multi-network reaching at 52 FPS runtime, ranked second based on the drivable area segmentation and lane line detection performance. For segmentation of dynamic objects, the network performance is increased by 22.45%, and considering mIoU overall performance increase is 3.96%. For a lidar sensor, a different modality is presented to detect objects. Two sensors' data are fused by proposed fusion algorithm, and results are tested and evaluated with the KITTI dataset. The proposed fusion methodology outperforms the stand-alone lidar methods about 3.58% and 3.63% on BEV and 3D detection MAP, respectively. Overall, benchmarking with two distinct fusion approaches illustrates the effectiveness of the proposed method.

Nighttime Traffic Object Detection Via Adaptively Integrating Event and Frame Domains

Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion

Sensor-Fused Nighttime System for Enhanced Pedestrian Detection in ADAS and Autonomous Vehicles

Multispectral Deep Neural Network Fusion Method for Low-Light Object Detection

Transformer fusion and histogram layer multispectral pedestrian detection network

Cross-modality feature fusion for night pedestrian detection

A Fast RetinaNet Fusion Framework for Multi-Spectral Pedestrian Detection

Beyond Night Visibility: Adaptive Multi-Scale Fusion of Infrared and Visible Images

Nighttime Pedestrian Detection Based on a Fusion of Visual Information and Millimeter-Wave Radar

Object Detection Using Multi-Sensor Fusion Based on Deep Learning

Lightweight Real-Time Object Detection via Enhanced Global Perception and Intra-Layer Interaction for Complex Traffic Scenarios

Embracing Events and Frames with Hierarchical Feature Refinement Network for Object Detection

3D Vehicle Detection Using Multi-Level Fusion From Point Clouds and Images

Late sensor fusion approach with a designed multi-segmentation network

Traffic Light Recognition for Complex Scene with Fusion Detections.

RGB-X Object Detection via Scene-Specific Fusion Modules

Multi-Modal Fusion Based on Depth Adaptive Mechanism for 3D Object Detection

MMAF-Net: Multi-view multi-stage adaptive fusion for multi-sensor 3D object detection

Let There be Light: Improved Traffic Surveillance via Detail Preserving Night-to-Day Transfer

Research on 24-Hour Dense Crowd Counting and Object Detection System Based on Multimodal Image Optimization Feature Fusion

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera