Abstract:Perceiving the surrounding environment is a fundamental task in autonomous driving. To obtain highly accurate perception results, modern autonomous driving systems typically employ multi-modal sensors to collect comprehensive environmental data. Among these, the radar-camera multi-modal perception system is especially favored for its excellent sensing capabilities and cost-effectiveness. However, the substantial modality differences between radar and camera sensors pose challenges in fusing information. To address this problem, this paper presents RCBEVDet, a radar-camera fusion 3D object detection framework. Specifically, RCBEVDet is developed from an existing camera-based 3D object detector, supplemented by a specially designed radar feature extractor, RadarBEVNet, and a Cross-Attention Multi-layer Fusion (CAMF) module. Firstly, RadarBEVNet encodes sparse radar points into a dense bird's-eye-view (BEV) feature using a dual-stream radar backbone and a Radar Cross Section aware BEV encoder. Secondly, the CAMF module utilizes a deformable attention mechanism to align radar and camera BEV features and adopts channel and spatial fusion layers to fuse them. To further enhance RCBEVDet's capabilities, we introduce RCBEVDet++, which advances the CAMF through sparse fusion, supports query-based multi-view camera perception models, and adapts to a broader range of perception tasks. Extensive experiments on the nuScenes show that our method integrates seamlessly with existing camera-based 3D perception models and improves their performance across various perception tasks. Furthermore, our method achieves state-of-the-art radar-camera fusion results in 3D object detection, BEV semantic segmentation, and 3D multi-object tracking tasks. Notably, with ViT-L as the image backbone, RCBEVDet++ achieves 72.73 NDS and 67.34 mAP in 3D object detection without test-time augmentation or model ensembling.

RCBEVDet: Radar-camera Fusion in Bird’s Eye View for 3D Object Detection

RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection

RCBEVDet++: Toward High-accuracy Radar-Camera Fusion 3D Perception Network

BEV-Radar: Bidirectional Radar-Camera Fusion for 3D Object Detection

RCFusion: Fusing 4-D Radar and Camera with Bird's-Eye View Features for 3-D Object Detection.

HVDetFusion: A Simple and Robust Camera-Radar Fusion Framework

Bridging the View Disparity Between Radar and Camera Features for Multi-Modal Fusion 3D Object Detection

RaCFormer: Towards High-Quality 3D Object Detection via Query-based Radar-Camera Fusion

UniBEVFusion: Unified Radar-Vision BEVFusion for 3D Object Detection

IRBEVF-Q: Optimization of Image-Radar Fusion Algorithm Based on Bird's Eye View Features

RCM-Fusion: Radar-Camera Multi-Level Fusion for 3D Object Detection

KAN-RCBEVDepth: A multi-modal fusion algorithm in object detection for autonomous driving

Traffic Object Detection for Autonomous Driving Fusing LiDAR and Pseudo 4D-Radar under Bird’s-Eye-View

BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View

SimpleBEV: Improved LiDAR-Camera Fusion Architecture for 3D Object Detection

SemanticBEVFusion: Rethink LiDAR-Camera Fusion in Unified Bird's-Eye View Representation for 3D Object Detection

CenterRadarNet: Joint 3D Object Detection and Tracking Framework using 4D FMCW Radar

HGSFusion: Radar-Camera Fusion with Hybrid Generation and Synchronization for 3D Object Detection

A Resource Efficient Fusion Network for Object Detection in Bird's-Eye View using Camera and Raw Radar Data

BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregation

CR3DT: Camera-RADAR Fusion for 3D Detection and Tracking