Abstract:Depth estimation from images serves as the fundamental step of 3D perception for autonomous driving and is an economical alternative to expensive depth sensors like LiDAR. The temporal photometric constraints enables self-supervised depth estimation without labels, further facilitating its application. However, most existing methods predict the depth solely based on each monocular image and ignore the correlations among multiple surrounding cameras, which are typically available for modern self-driving vehicles. In this paper, we propose a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras. Specifically, we employ a joint network to process all the surrounding views and propose a cross-view transformer to effectively fuse the information from multiple views. We apply cross-view self-attention to efficiently enable the global interactions between multi-camera feature maps. Different from self-supervised monocular depth estimation, we are able to predict real-world scales given multi-camera extrinsic matrices. To achieve this goal, we adopt the two-frame structure-from-motion to extract scale-aware pseudo depths to pretrain the models. Further, instead of predicting the ego-motion of each individual camera, we estimate a universal ego-motion of the vehicle and transfer it to each view to achieve multi-view ego-motion consistency. In experiments, our method achieves the state-of-the-art performance on the challenging multi-camera depth estimation datasets DDAD and nuScenes.

Decoupling Relative Pose Estimation Method for Non-Overlapping Multi-Camera System

Efficient Multi-person Hierarchical 3D Pose Estimation for Autonomous Driving

Robust And Accurate Multiple-Camera Pose Estimation Toward Robotic Applications

Global Pose Estimation Iterative Algorithm for Multi-camera from Point and Line Correspondences

Realization of odometry system using monocular vision

Pose Measurement of Non-Cooperative Target Based on Stereo Vision

Robust and Efficient Vehicles Motion Estimation with Low-Cost Multi-Camera and Odometer-Gyroscope

Relative Pose Estimation for Multi-Camera Systems from Point Correspondences with Scale Ratio

An Efficient Method to Recover Relative Pose for Vehicle-Mounted Cameras under Planar Motion.

Globally Optimal Relative Pose Estimation for Multi-Camera Systems with Known Gravity Direction

Pose measurement method for non-cooperative space vehicle using multiple non-overlapping cameras

Planar Motion Estimation for Multi-camera System.

A 2-Point Pose Estimation Algorithm For Monocular Visual Odometry Of Ground Vehicles

Partially Calibrated Camera Based Pose Estimation Of Mobile Robots With Application To Visual Tracking

An Accurate and Real-time Relative Pose Estimation from Triple Point-line Images by Decoupling Rotation and Translation

Estimation of Vehicle Pose and Position with Monocular Camera at Urban Road Intersections

Relative Pose for Nonrigid Multi-Perspective Cameras: The Static Case

SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation

Monocular 3D Object Detection: An Extrinsic Parameter Free Approach

A Relative Pose Estimation Method of Non-Cooperative Space Targets

Affine Correspondences between Multi-Camera Systems for Relative Pose Estimation