Abstract:Object detection forms the foundation of safe autonomous vehicle (AV) operation. LiDAR and camera are both widely used detection devices, yet they each come with their unique advantages and drawbacks. For instance, LiDAR sensors face challenges such as obstacle occlusion and long-range object detection when applied to 3-D object recognition. On the other hand, cameras are significantly affected by variations in lighting and weather conditions, and they struggle to provide precise depth information. Hence, multisensor fusion is frequently employed to enhance both the accuracy and robustness of object detection. Prominent issues associated with end-to-end fusion include feature misalignment and suboptimal training strategies, while the challenge for the sequential fusion architecture lies in its inability to fully tap into the capabilities of high-density images to enhance point cloud data, especially when dealing with information sparsity at extended ranges. To address these challenges, we present a dense sequential fusion (DSF) framework specifically designed to fuse camera and LiDAR sensor data. The primary goal is to enhance the accuracy and robustness of 3-D object detection, particularly for distant objects. First, we developed a model for augmenting foreground points, specifically targeting sparse points associated with far-range objects. Second, a foreground points refinement technique was implemented to filter long-tail points generated by images. This refinement process has the capability to improve the object’s distinctiveness, especially when dealing with an abundance of edge points while also supplying high-resolution raw and pseudo foreground points. Finally, voxel-based LiDAR 3-D detection methods were employed to detect 3-D objects utilizing the high-resolution raw and pseudo point clouds. The experimental studies were conducted using the KITTI dataset. The results showed that the proposed method improved 3-D mAP by 2.59% compared with PointPillars and 1.27% average precision (AP) for car hard-level detection compared with SECOND. Furthermore, it improved the bird’s eye view (BEV) AP for far-range car detection by more than 10%.

Research on 3D Object Detection Based on Laser Point Cloud and Image Fusion

LiDAR-Camera Cross Fusion Network Towards 3D Object Detection in Self-Driving

Research on 3D Point Cloud Object Detection Algorithm for Autonomous Driving

Three-Dimensional Object Detection Network Based on Multi-Layer and Multi-Modal Fusion

RI-Fusion: 3D Object Detection Using Enhanced Point Features With Range-Image Fusion for Autonomous Driving.

Three-Dimensional Point Cloud Object Detection Based on Feature Fusion and Enhancement

3D Object Detection Based on Attention and Multi-Scale Feature Fusion

PI-RCNN: An Efficient Multi-sensor 3D Object Detector with Point-based Attentive Cont-conv Fusion Module

FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving

A LiDAR–Camera Fusion 3D Object Detection Algorithm

Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving

FusionRCNN: LiDAR-Camera Fusion for Two-stage 3D Object Detection

FFPA-Net: Efficient Feature Fusion with Projection Awareness for 3D Object Detection

3D object detection based on fusion of image and point cloud in autonomous driving traffic scenarios

One-Stage Multi-Sensor Data Fusion Convolutional Neural Network For 3d Object Detection

Cascade fusion of multi-modal and multi-source feature fusion by the attention for three-dimensional object detection

3DMMF: 3D Object Detection Network Based on Multi-Layer and Multi-Modal Fusion

Anti-Noise 3D Object Detection of Multimodal Feature Attention Fusion Based on PV-RCNN

3D Small Object Detection from Cameras and Point Clouds Using Five-Head Attention in a Fusion Method

3D Vehicle Detection Using Multi-Level Fusion From Point Clouds and Images

Dense Sequential Fusion: Point Cloud Enhancement Using Foreground Mask Guidance for Multimodal 3-D Object Detection