Abstract:3D object detection is a challenging and promising task for autonomous driving and robotics, benefiting significantly from multi-sensor fusion, such as LiDAR and cameras. Conventional methods for sensor fusion rely on a projection matrix to align the features from LiDAR and cameras. However, these methods often suffer from inadequate flexibility and robustness, leading to lower alignment accuracy under complex environmental conditions. Addressing these challenges, in this paper, we propose a novel Bidirectional Attention Fusion module, named BAFusion,which effectively fuses the information from LiDAR and cameras using cross-attention. Unlike the conventional methods, our BAFusion module can adaptively learn the cross-modal attention weights, making the approach more flexible and robust. Moreover, drawing inspiration from advanced attention optimization techniques in 2D vision, we developed the Cross Focused Linear Attention Fusion Layer (CFLAF Layer) and integrated it into our BAFusion pipeline. This layer optimizes the computational complexity of attention mechanisms and facilitates advanced interactions between image and point cloud data, showcasing a novel approach to addressing the challenges of cross-modal attention calculations. We evaluated our method on the KITTI dataset using various baseline networks, such as PointPillars, SECOND, and Part-A2, and demonstrated consistent improvements in 3D object detection performance over these baselines, especially for smaller objects like cyclists and pedestrians. Our approach achieves competitive results on the KITTI benchmark.

Contrastive Late Fusion for 3D Object Detection

Three-Dimensional Object Detection Network Based on Multi-Layer and Multi-Modal Fusion

Rethinking the Late Fusion of LiDAR-Camera Based 3D Object Detection

GOOD: General Optimization-based Fusion for 3D Object Detection via LiDAR-Camera Object Candidates

LiDAR-Camera Cross Fusion Network Towards 3D Object Detection in Self-Driving

DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection

Cascaded Cross-Modality Fusion Network for 3D Object Detection

FGFusion: Fine-Grained Lidar-Camera Fusion for 3D Object Detection

LEF: Late-to-Early Temporal Fusion for LiDAR 3D Object Detection

3DMMF: 3D Object Detection Network Based on Multi-Layer and Multi-Modal Fusion

MSGFusion: Muti-scale Semantic Guided LiDAR-Camera Fusion for 3D Object Detection

Explore the LiDAR-Camera Dynamic Adjustment Fusion for 3D Object Detection

FFPA-Net: Efficient Feature Fusion with Projection Awareness for 3D Object Detection

BAFusion: Bidirectional Attention Fusion for 3D Object Detection Based on LiDAR and Camera

AFMCT: adaptive fusion module based on cross-modal transformer block for 3D object detection

FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving

RangeLVDet: Boosting 3D Object Detection in LIDAR With Range Image and RGB Image

MMLF: Multi-modal Multi-class Late Fusion for Object Detection with Uncertainty Estimation

ContrastAlign: Toward Robust BEV Feature Alignment via Contrastive Learning for Multi-Modal 3D Object Detection

Enhancing 3D object detection through multi-modal fusion for cooperative perception

Multi-Modal Fusion Based on Depth Adaptive Mechanism for 3D Object Detection