Abstract:Object detection forms the foundation of safe autonomous vehicle (AV) operation. LiDAR and camera are both widely used detection devices, yet they each come with their unique advantages and drawbacks. For instance, LiDAR sensors face challenges such as obstacle occlusion and long-range object detection when applied to 3-D object recognition. On the other hand, cameras are significantly affected by variations in lighting and weather conditions, and they struggle to provide precise depth information. Hence, multisensor fusion is frequently employed to enhance both the accuracy and robustness of object detection. Prominent issues associated with end-to-end fusion include feature misalignment and suboptimal training strategies, while the challenge for the sequential fusion architecture lies in its inability to fully tap into the capabilities of high-density images to enhance point cloud data, especially when dealing with information sparsity at extended ranges. To address these challenges, we present a dense sequential fusion (DSF) framework specifically designed to fuse camera and LiDAR sensor data. The primary goal is to enhance the accuracy and robustness of 3-D object detection, particularly for distant objects. First, we developed a model for augmenting foreground points, specifically targeting sparse points associated with far-range objects. Second, a foreground points refinement technique was implemented to filter long-tail points generated by images. This refinement process has the capability to improve the object’s distinctiveness, especially when dealing with an abundance of edge points while also supplying high-resolution raw and pseudo foreground points. Finally, voxel-based LiDAR 3-D detection methods were employed to detect 3-D objects utilizing the high-resolution raw and pseudo point clouds. The experimental studies were conducted using the KITTI dataset. The results showed that the proposed method improved 3-D mAP by 2.59% compared with PointPillars and 1.27% average precision (AP) for car hard-level detection compared with SECOND. Furthermore, it improved the bird’s eye view (BEV) AP for far-range car detection by more than 10%.

A Multimodal Fusion System for Object Identification in Point Clouds with Density and Coverage Differences

A Novel Multimodal Fusion Framework Based on Point Cloud Registration for Near-Field 3D SAR Perception

Multi-Modal Fusion Based on Depth Adaptive Mechanism for 3D Object Detection

Dense Sequential Fusion: Point Cloud Enhancement Using Foreground Mask Guidance for Multimodal 3-D Object Detection

A Quantitative Validation of Multi-Modal Image Fusion and Segmentation for Object Detection and Tracking

A Multi-Level Eigenvalue Fusion Algorithm for 3D Multi-Object Tracking

mmFUSION: Multimodal Fusion for 3D Objects Detection

Center Feature Fusion: Selective Multi-Sensor Fusion of Center-based Objects

Muti-Frame Point Cloud Feature Fusion Based on Attention Mechanisms for 3D Object Detection

HybridFusion: LiDAR and Vision Cross-Source Point Cloud Fusion

IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection

Virtual Namesake Point Multi-Source Point Cloud Data Fusion Based on FPFH Feature Difference

ACF-Net: Asymmetric Cascade Fusion for 3D Detection with LiDAR Point Clouds and Images

MFFNet: Multimodal Feature Fusion Network for Point Cloud Semantic Segmentation

3D Point Cloud Fusion Method Based on EMD Auto-Evolution and Local Parametric Network

PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest

Point Cloud Densification Algorithm for Multiple Cameras and Lidars Data Fusion

Structure-Aware Multimodal Feature Fusion for RGB-D Scene Classification and Beyond

Cascade fusion of multi-modal and multi-source feature fusion by the attention for three-dimensional object detection

MixedFusion: 6D Object Pose Estimation from Decoupled RGB-Depth Features.

Three-Dimensional Point Cloud Object Detection Based on Feature Fusion and Enhancement