Abstract:Omnidirectional depth estimation is an emerging research topic and has received significant attention in recent years. However, the existing methods were developed based on the theory of planar stereo matching; and introduce the nonlinear epipolar constraint and significant distortions of re-projections. In this paper, we propose a novel approach that use spherical CNNs and the epipolar constraint on sphere for omnidirectional depth estimation. We discuss the epipolar constraint for spherical stereo imaging and convert the nonlinear constraint on a planar projection to the linear constraint on a sphere. We then propose a Spherical Convolution Residual Network (SCRN) for omnidirectional depth estimation via the spherical linear epipolar constraint. The input equirectangular projection (ERP) images are sampled to spherical meshes and fed into SCRN to calculate spherical depth maps. For 2D visualization, we design a Planar Refinement Network (PRN) and adopt the cascade learning scheme to improve the accuracy of depth maps. This scheme reduces the errors caused by projection, interpolation, and the limitation of spherical representation. The experiment shows that our full scheme Cascade Spherical Depth Network (CSDNet) results in more accurate and detailed depth maps with lower errors, as compared to recent seminal works. Our approach yields the comparable performance to the other state-of-the-art works on the omnidirectional stereo datasets with less number of parameters. The effectiveness of the spherical network and the cascade learning scheme is validated, and the influence of spherical sampling density is also discussed. (c) 2021 Elsevier B.V. All rights reserved.

Direct Triangulation with Spherical Projection for Omnidirectional Cameras

Multi-view 3D Reconstruction from Uncalibrated Radially-Symmetric Cameras

Calibration of Central Omnidirectional Cameras Via the Viewing Sphere

Stereo Calibration and Rectification for Omnidirectional Multi-Camera Systems

Unwrapping and Stereo Rectification for Omnidirectional Images

Terrain Reconstruction of Lunar Surface Based on Binocular Fisheye Camera

A unified rectification method for single viewpoint multi-camera system

Camera Calibration with Spheres: Linear Approaches

Spherical Pseudo-Cylindrical Representation for Omnidirectional Image Super-resolution

A Spherical Rectification for Dual-PTZ-Camera System

3D Reconstruction of Spherical Images based on Incremental Structure from Motion

Omnidirectional stereo depth estimation based on spherical deep network

Linear Catadioptric Camera Calibration from Sphere Images

Group geometric calibration and rectification for circular multi-camera imaging system.

Omnidirectional 3D Reconstruction Using Rotating Camera with Mirrors

Space and camera path reconstruction for omni-directional vision

Hybrid Stereo Configurations Through a Cylindrical Sensor Calibration

Full Camera Calibration from a Single View of Planar Scene

Linear Approaches to Camera Calibration from Sphere Images or Active Intrinsic Calibration Using Vanishing Points

3D Scene Geometry Estimation from 360$^\circ$ Imagery: A Survey