Abstract:Three dimensional (3D) object detection using the camera and light detection and ranging (LiDAR) fusion model has received much attention for meeting the environmental sensing requirement in autonomous vehicles. Due to the intrinsic data difference and the uneven distribution of LiDAR data, integrating the camera and LiDAR to solve the problem of the uneven spatial distribution of LiDAR data is still challenging. This paper proposes a novel 3D detector, which decorates the point cloud voxels with the semantic features at multi-scale and fuses the multi-source features by cross-attention to improve the detection performance. Specifically, the multi-modal fusion aims to maintain the cross-modal consistency by using the frustum model to construct the correspondence between point voxels and image pixels at multi-scale. To fully exploit the geometric constraints of voxels, we developed a adaptive sampling radius module to dynamically select the sampling radius of the voxel set abstraction module based on softmax. Moreover, we propose a multi-source fusion module which utilizes the cross-attention and takes the raw point cloud's spatial distribution as the clue to fuse the features of point clouds, bird's eye view information and the aggregated voxels to obtain the multi-scale and multi-source fusion features. Finally, region proposal network is adopted to generate and refine the 3D bbox and class prediction based on the fusion feature and bird's eye view feature. Extensive experiments are conducted on the two publicly available 3D object datasets proposed by Karlsruhe Institute of technology and Toyota Technological Institute (KITTI) and the nuScenes. The proposed model reaches 86.27% mean Average Precision (mAP) for car on KITTI and obtains 6.56% gains than PointPainting. Moreover, our model reaches 70.70% mAP on nuScens and improves 2.61% than Transfusion. Furthermore, comprehensive ablation experiments are conducted to validate the effects and contribution of the different components of the proposed model.

Cascading Classifier with Discriminative Multi-Features for a Specific 3D Object Real-Time Detection.

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

3D Cascade RCNN: High Quality Object Detection in Point Clouds

SPV-SSD: An Anchor-Free 3D Single-Stage Detector with Supervised-PointRendering and Visibility Representation

CascadeV-Det: Cascade Point Voting for 3D Object Detection

CasA: A Cascade Attention Network for 3-D Object Detection from LiDAR Point Clouds

Online learning of cascaded classifier designed for multi-object tracking

Cascaded Cross-Modality Fusion Network for 3D Object Detection

Multi-feature Enhancement Based on Sparse Networks for Single-Stage 3D Object Detection

Cascaded Multi-3D-view Fusion for 3D-Oriented Object Detection

Cascaded Matching Based on Detection Box Area for Multi-Object Tracking

Multi-Modality Task Cascade for 3D Object Detection

Cascaded Iterative Training Model and Parallel Multi-Classifiers for Visual Object Tracking

Cascade fusion of multi-modal and multi-source feature fusion by the attention for three-dimensional object detection

Structure Aware Single-Stage 3D Object Detection From Point Cloud

Effective 3D Object Detection Based on Detector and Tracker.

Real-time 3D object proposal generation and classification under limited processing resources

Real-Time Detection for Camera Sensing System: Adaptive Cascade Single-Shot Detector

Efficient Point-Based Single Scale 3D Object Detection from Traffic Scenes.

Monocular 3D Detection for Autonomous Vehicles by Cascaded Geometric Constraints and Depurated Using 3D Results

DCDet: Dynamic Cross-based 3D Object Detector