Abstract:The 3D object detection is becoming indispensable for environmental perception in autonomous driving. Light detection and ranging (LiDAR) point clouds often fail to distinguish objects with similar structures and are quite sparse for distant or small objects, thereby introducing false and missed detections. To address these issues, LiDAR is often fused with cameras due to the rich textural information provided by images. However, current fusion methods suffer the inefficient data representation and inaccurate alignment of heterogeneous features, leading to poor precision and low efficiency. To this end, we propose a plug-and-play module termed range-image fusion (RI-Fusion) to achieve an effective fusion of LiDAR and camera data, designed to be easily accessible by existing mainstream LiDAR-based algorithms. In this process, we design an image and point cloud alignment method by converting a point cloud into a compact range-view representation through a spherical coordinate transformation. The range image is then integrated with a corresponding camera image utilizing an attention mechanism. The original range image is then concatenated with fusion features to retain point cloud information, and the results are projected onto a spatial point cloud. Finally, the feature-enhanced point cloud can be input into a LiDAR-based 3D object detector. The results of validation experiments involving the KITTI 3D object detection benchmark showed that our proposed fusion method significantly enhanced multiple mainstream LiDAR-based 3D object detectors, PointPillars, SECOND, and Part A(2), improving the 3D mAP (mean Average Precision) by 3.61%, 2.98%, and 1.27%, respectively, particularly for small objects such as pedestrians and cyclists.

Road Object Detection Using a Disparity-Based Fusion Model

A Reinforced Road Detection Method in Complicated Environment

A Multi-view 3D Vehicle Detection Method Based On Novel 3D Proposal Generation Method

RGB-LiDAR fusion for accurate 2D and 3D object detection

Enhancing 3D object detection through multi-modal fusion for cooperative perception

Color Vision-Based Multi-Level Analysis and Fusion for Road Area Detection

Multi-Modal Fusion Based on Depth Adaptive Mechanism for 3D Object Detection

Detection-level Fusion for Multi-Object Perception in Dense Traffic Environment

Robust Object Detection with Real-Time Fusion of Multiview Foreground Silhouettes

3D Instance Segmentation and Object Detection Framework Based on the Fusion of Lidar Remote Sensing and Optical Image Sensing

Real-Time Vehicle Object Detection Method Based on Multi-Scale Feature Fusion

Occlusion-Guided Multi-Modal Fusion for Vehicle-Infrastructure Cooperative 3D Object Detection

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

3D Vehicle Detection Using Multi-Level Fusion From Point Clouds and Images

Road Scene Segmentation Via Fusing Camera and Lidar Data.

MRD-YOLO: A Multispectral Object Detection Algorithm for Complex Road Scenes

RI-Fusion: 3D Object Detection Using Enhanced Point Features With Range-Image Fusion for Autonomous Driving.

Complementary Features With Reasonable Receptive Field For Road Scene 3d Object Detection

MicroRNA expression analysis and Multiplex ligation‐dependent probe amplification in metastatic and non‐metastatic uveal melanoma

RangeLVDet: Boosting 3D Object Detection in LIDAR With Range Image and RGB Image