Abstract:Detecting objects in 3D space using multiple cameras, known as Multi-Camera 3D Object Detection (MC3D-Det), has gained prominence with the advent of bird's-eye view (BEV) approaches. However, these methods often struggle when faced with unfamiliar testing environments due to the lack of diverse training data encompassing various viewpoints and environments. To address this, we propose a novel method that aligns 3D detection with 2D camera plane results, ensuring consistent and accurate detections. Our framework, anchored in perspective debiasing, helps the learning of features resilient to domain shifts. In our approach, we render diverse view maps from BEV features and rectify the perspective bias of these maps, leveraging implicit foreground volumes to bridge the camera and BEV planes. This two-step process promotes the learning of perspective- and context-independent features, crucial for accurate object detection across varying viewpoints, camera parameters, and environmental conditions. Notably, our model-agnostic approach preserves the original network structure without incurring additional inference costs, facilitating seamless integration across various models and simplifying deployment. Furthermore, we also show our approach achieves satisfactory results in real data when trained only with virtual datasets, eliminating the need for real scene annotations. Experimental results on both Domain Generalization (DG) and Unsupervised Domain Adaptation (UDA) clearly demonstrate its effectiveness. The codes are available at <a class="link-external link-https" href="https://github.com/EnVision-Research/Generalizable-BEV" rel="external noopener nofollow">this https URL</a>.

Perspective-Equivariance for Unsupervised Imaging with Camera Geometry

Towards Generalizable Multi-Camera 3D Object Detection via Perspective Debiasing

A Perspective on Deep Imaging

Perspective Fields for Single Image Camera Calibration

3D Reconstruction through Fusion of Cross-View Images

Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

Viewpoint Equivariance for Multi-View 3D Object Detection

The Right (Angled) Perspective: Improving the Understanding of Road Scenes Using Boosted Inverse Perspective Mapping

Deep Perspective Transformation Based Vehicle Localization on Bird's Eye View

Equivariant Light Field Convolution and Transformer

Fully Unsupervised Dynamic MRI Reconstruction via Diffeo-Temporal Equivariance

Sketched Equivariant Imaging Regularization and Deep Internal Learning for Inverse Problems

UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations

Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging

Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection

Perspectively Equivariant Keypoint Learning for Omnidirectional Images

A Geometric Approach to Obtain a Bird's Eye View from an Image

Möbius Transform for Mitigating Perspective Distortions in Representation Learning

Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation

A geometry-informed deep learning framework for ultra-sparse 3D tomographic image reconstruction

Discrete Latent Perspective Learning for Segmentation and Detection