Abstract:Roadside camera-driven 3D object detection is a crucial task in intelligent transportation systems, which extends the perception range beyond the limitations of vision-centric vehicles and enhances road safety. While previous studies have limitations in using only depth or height information, we find both depth and height matter and they are in fact complementary. The depth feature encompasses precise geometric cues, whereas the height feature is primarily focused on distinguishing between various categories of height intervals, essentially providing semantic context. This insight motivates the development of Complementary-BEV (CoBEV), a novel end-to-end monocular 3D object detection framework that integrates depth and height to construct robust BEV representations. In essence, CoBEV estimates each pixel's depth and height distribution and lifts the camera features into 3D space for lateral fusion using the newly proposed two-stage complementary feature selection (CFS) module. A BEV feature distillation framework is also seamlessly integrated to further enhance the detection accuracy from the prior knowledge of the fusion-modal CoBEV teacher. We conduct extensive experiments on the public 3D detection benchmarks of roadside camera-based DAIR-V2X-I and Rope3D, as well as the private Supremind-Road dataset, demonstrating that CoBEV not only achieves the accuracy of the new state-of-the-art, but also significantly advances the robustness of previous methods in challenging long-distance scenarios and noisy camera disturbance, and enhances generalization by a large margin in heterologous settings with drastic changes in scene and camera parameters. For the first time, the vehicle AP score of a camera model reaches 80% on DAIR-V2X-I in terms of easy mode. The source code will be made publicly available at <a class="link-external link-https" href="https://github.com/MasterHow/CoBEV" rel="external noopener nofollow">this https URL</a>.

The Relationship Between a Urinary Cachectic Factor and Weight Loss in Advanced Cancer Patients

CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity

BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection

BEVHeight++: Toward Robust Visual Centric 3D Object Detection

BEVDepth: Acquisition of Reliable Depth for Multi-View 3D Object Detection

BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View

OCBEV: Object-Centric BEV Transformer for Multi-View 3D Object Detection

DA-BEV: Depth Aware BEV Transformer for 3D Object Detection

KAN-RCBEVDepth: A multi-modal fusion algorithm in object detection for autonomous driving

OA-BEV: Bringing Object Awareness to Bird's-Eye-View Representation for Multi-Camera 3D Object Detection

SA-BEV: Generating Semantic-Aware Bird's-Eye-View Feature for Multi-view 3D Object Detection

RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection

BEVScope: Enhancing Self-Supervised Depth Estimation Leveraging Bird's-Eye-View in Dynamic Scenarios

CoDRMA: Collaborative Depth Refinement Via Dual-Mask and Dual-Attention for Bird’s Eye View Collaborative 3D Object Detection

TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning

Co-Fix3D: Enhancing 3D Object Detection with Collaborative Refinement

CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers

UniBEV: Multi-modal 3D Object Detection with Uniform BEV Encoders for Robustness against Missing Sensor Modalities

RCBEVDet: Radar-camera Fusion in Bird’s Eye View for 3D Object Detection