Abstract:Multiple-view stereo has potential applications in robotic operations and autonomous driving (unstructured environment construction, visual servo). With assisted depth information, inertial navigation systems can achieve precise navigation. It is, especially suitable for GPS failures in complex environments. Accurate depth estimation is a challenge in low-textured or occluded regions. To alleviate the inference of incorrect depth, a multi-stage pixel-visibility learning-based stereo network is presented in this paper. Its improvements are as follows: 1) a new content-adaptive cost volume aggregation mechanism based on neighboring pixel-wise visibility is designed to effectively produce more accurate and smoother depth map predictions in the object boundary. 2) global convolution block and boundary refinement block are developed to regularize its cost volume, they can learn the inherent constraints of feature matching correspondence and effectively mitigate the depth estimation uncertainty in low-textured regions. 3) a new loss function is designed to measure the uncertainty of predicted probability distribution and enhance the reliability of depth map inference. Experimental results on the indoor DTU datasets and the outdoor Tanks & Temples datasets indicate that our method can achieve superior performance and has a powerful generalization ability, which is comparable to state-of-the-art works. Note to Practitioners—Multiple-view stereo (MVS) can estimate dense 3D representations of scenes, which is widely used in autonomous driving, robotic navigation, virtual reality (VR), and augmented reality (AR). Aiming at the problem of incorrect depth inference in low-textured or occluded regions, this work proposes a novel multi-stage depth prediction method based on neighboring pixel-wise visibility. Our method cannot only achieve accurate depth estimation for robot perception but also make no concession to real-time performance. It is clear that the proposed method has good potential in 3D reconstruction, robotic navigation, and VR/AR fields to provide accurate depth estimation in real-time with limited memory consumption.

A Multiview Stereo Algorithm Based on Image Segmentation Guided Generation of Planar Prior for Textureless Regions of Artificial Scenes

Prior surface model guided dense reconstruction via graph-cuts

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

High Fidelity Aggregated Planar Prior Assisted PatchMatch Multi-View Stereo

HQP-MVS:High-Quality Plane Priors Assisted Multi-View Stereo for Low-Textured Areas

Confidence-Guided Planar-Recovering Multiview Stereo for Weakly Textured Plane of High-Resolution Image Scenes

High-Quality Depth Recovery Via Interactive Multi-view Stereo

PlaneStereo: Plane-aware Multi-view Stereo

3d Stereoscopic Image Generation From Monocular View In Structured Environments

Segmentation-aware Prior Assisted Joint Global Information Aggregated 3D Building Reconstruction

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

LGP-MVS: Combined Local and Global Planar Priors Guidance for Indoor Multi-View Stereo

MSP-MVS: Multi-granularity Segmentation Prior Guided Multi-View Stereo

High completeness multi-view stereo for dense reconstruction of large-scale urban scenes

A Superpixel Method Using 3-D Geometry and Normal Priori Information for RGB-D Data

Multistage Pixel-Visibility Learning with Cost Regularization for Multiview Stereo

P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

Multi-View Depth Estimation with Color-Aware Propagation and Texture-Aware Triangulation

Depth-aware Volume Attention for Texture-less Stereo Matching

Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priors

Depth Estimation from Multi-Scale SLIC Superpixels Using Non-Parametric Learning