Abstract:Object detection forms the foundation of safe autonomous vehicle (AV) operation. LiDAR and camera are both widely used detection devices, yet they each come with their unique advantages and drawbacks. For instance, LiDAR sensors face challenges such as obstacle occlusion and long-range object detection when applied to 3-D object recognition. On the other hand, cameras are significantly affected by variations in lighting and weather conditions, and they struggle to provide precise depth information. Hence, multisensor fusion is frequently employed to enhance both the accuracy and robustness of object detection. Prominent issues associated with end-to-end fusion include feature misalignment and suboptimal training strategies, while the challenge for the sequential fusion architecture lies in its inability to fully tap into the capabilities of high-density images to enhance point cloud data, especially when dealing with information sparsity at extended ranges. To address these challenges, we present a dense sequential fusion (DSF) framework specifically designed to fuse camera and LiDAR sensor data. The primary goal is to enhance the accuracy and robustness of 3-D object detection, particularly for distant objects. First, we developed a model for augmenting foreground points, specifically targeting sparse points associated with far-range objects. Second, a foreground points refinement technique was implemented to filter long-tail points generated by images. This refinement process has the capability to improve the object’s distinctiveness, especially when dealing with an abundance of edge points while also supplying high-resolution raw and pseudo foreground points. Finally, voxel-based LiDAR 3-D detection methods were employed to detect 3-D objects utilizing the high-resolution raw and pseudo point clouds. The experimental studies were conducted using the KITTI dataset. The results showed that the proposed method improved 3-D mAP by 2.59% compared with PointPillars and 1.27% average precision (AP) for car hard-level detection compared with SECOND. Furthermore, it improved the bird’s eye view (BEV) AP for far-range car detection by more than 10%.

Boosting Cnn-Based Pedestrian Detection Via 3d Lidar Fusion In Autonomous Driving

Online Multipedestrian Tracking Based on Fused Detections of Millimeter Wave Radar and Vision

Fully convolutional neural networks for LIDAR–camera fusion for pedestrian detection in autonomous vehicle

A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

A Pedestrian Detection Algorithm Based on Score Fusion for Multi-LiDAR Systems

Combining LiDAR Space Clustering and Convolutional Neural Networks for Pedestrian Detection

FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving

Pedestrian Detection Using Multi-Channel Visual Feature Fusion by Learning Deep Quality Model.

Density Enhancement-Based Long-Range Pedestrian Detection Using 3-D Range Data

Face recognition via discriminative dictionary learning

Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection

Dense Sequential Fusion: Point Cloud Enhancement Using Foreground Mask Guidance for Multimodal 3-D Object Detection

Leveraging Deep Convolutional Neural Networks Pre-Trained on Autonomous Driving Data for Vehicle Detection From Roadside LiDAR Data

Enhancing 3D object detection through multi-modal fusion for cooperative perception

6DoF-3D: Efficient and accurate 3D object detection using six degrees-of-freedom for autonomous driving

Influence of Camera-LiDAR Configuration on 3D Object Detection for Autonomous Driving

3D Object Detection for Point Cloud in Virtual Driving Environment

Pedestrian Detection with Multi-View Convolution Fusion Algorithm

Multispectral Deep Neural Networks for Pedestrian Detection

3D Vehicle Detection Using Multi-Level Fusion From Point Clouds and Images