Abstract:Multiple-View Stereo (MVS) aims to reconstruct the dense 3D representations of scenes. MVS has potential applications in the fields of autonomous driving (unstructured environment construction) and robotic navigation (visual-inertial navigation). To mitigate the error of depth estimation in low-textured or occluded regions, this work proposes a two-stage multi-view stereo network for fast and accurate depth estimation. The improvements of this work over the state of the art are as follows: 1) Sparse costs are constructed to jointly predict the initial depth map and surface normal by cost regularization, which proves that the surface normals can be estimated in this way with low memory consumption. 2) A new edge refinement block is developed to refine the coarse surface normal to obtain a fine-grained surface normal map. 3) Instead of using the general variance-based metric to equally aggregate cost, a new content-adaptive cost aggregation mechanism based on the similarity of the neighboring surface normal is designed for reliable cost aggregation. To the best of our knowledge, the proposed work is the first trainable network that leverages surface normal as guidance to capture neighboring pixel-visibility, which is an effective supplement to existing depth/normal estimation frameworks. Experimental results indicate that our method can not only achieve accurate depth estimation for scene perception but also make no concession to the real-time performance and limited memory bottleblock. Multiple-view stereo (MVS) aims to reconstruct the dense 3D representations of scenes. It is widely used in the fields of industrial measurement, autonomous driving, and robotic navigation. To mitigate the error of depth estimation in challenging scenarios, this work proposes a two-stage multi-view stereo network for fast and accurate depth estimation. Our method is the first trainable network that leverages surface normal as pixel-visibility guidance to aggregate reliable cost, which could achieve accurate depth estimation and provide the perception ability for the robot. The proposed method has great potential in the fields of 3D reconstruction, industrial measurement, and robotic navigation to estimate real-time and accurate depth with limited memory consumption.

Augmenting Depth Estimation from Deep Convolutional Neural Network Using Multi-Spectral Photometric Stereo

Three-Dimensional Reconstruction from Single Image Base on Combination of CNN and Multi-Spectral Photometric Stereo

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Depth Generation Network: Estimating Real World Depth From Stereo And Depth Images

Monocular Depth Estimation Based on Multi-Scale Graph Convolution Networks

Binocular Depth Estimation Using Convolutional Neural Network With Siamese Branches.

Depth Map Inpainting Using a Fully Convolutional Network

Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Normal Assisted Pixel-Visibility Learning with Cost Aggregation for Multiview Stereo

Surface Depth Estimation From Multiview Stereo Satellite Images With Distribution Contrast Network

Monocular Depth Estimation With Affinity, Vertical Pooling, And Label Enhancement

MEStereo-Du2CNN: A Novel Dual Channel CNN for Learning Robust Depth Estimates from Multi-exposure Stereo Images for HDR 3D Applications

Estimating High-resolution Surface Normals via Low-resolution Photometric Stereo Images

MEStereo-Du2CNN: a dual-channel CNN for learning robust depth estimates from multi-exposure stereo images for HDR 3D applications

Learning Conditional Photometric Stereo with High-Resolution Features

Monocular Depth Estimation with Guidance of Surface Normal Map

A deep-shallow and global–local multi-feature fusion network for photometric stereo

Unsupervised Depth Estimation from Light Field Using a Convolutional Neural Network

Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels