Abstract:Multiple-view stereo has potential applications in robotic operations and autonomous driving (unstructured environment construction, visual servo). With assisted depth information, inertial navigation systems can achieve precise navigation. It is, especially suitable for GPS failures in complex environments. Accurate depth estimation is a challenge in low-textured or occluded regions. To alleviate the inference of incorrect depth, a multi-stage pixel-visibility learning-based stereo network is presented in this paper. Its improvements are as follows: 1) a new content-adaptive cost volume aggregation mechanism based on neighboring pixel-wise visibility is designed to effectively produce more accurate and smoother depth map predictions in the object boundary. 2) global convolution block and boundary refinement block are developed to regularize its cost volume, they can learn the inherent constraints of feature matching correspondence and effectively mitigate the depth estimation uncertainty in low-textured regions. 3) a new loss function is designed to measure the uncertainty of predicted probability distribution and enhance the reliability of depth map inference. Experimental results on the indoor DTU datasets and the outdoor Tanks & Temples datasets indicate that our method can achieve superior performance and has a powerful generalization ability, which is comparable to state-of-the-art works. Note to Practitioners—Multiple-view stereo (MVS) can estimate dense 3D representations of scenes, which is widely used in autonomous driving, robotic navigation, virtual reality (VR), and augmented reality (AR). Aiming at the problem of incorrect depth inference in low-textured or occluded regions, this work proposes a novel multi-stage depth prediction method based on neighboring pixel-wise visibility. Our method cannot only achieve accurate depth estimation for robot perception but also make no concession to real-time performance. It is clear that the proposed method has good potential in 3D reconstruction, robotic navigation, and VR/AR fields to provide accurate depth estimation in real-time with limited memory consumption.

Self-Supervised Multi-View Stereo with Adaptive Depth Priors

Self-supervised Multi-view Stereo Via Inter and Intra Network Pseudo Depth

Sparse Prior Guided Deep Multi-View Stereo

Unsupervised multi-view stereo network based on multi-stage depth estimation

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Multi-View Stereo Representation Revist: Region-Aware MVSNet

Self-supervised Multi-view Stereo Via Effective Co-Segmentation and Data-Augmentation.

Digging into Uncertainty in Self-supervised Multi-view Stereo

Hierarchical Prior Mining for Non-local Multi-View Stereo.

Multistage Pixel-Visibility Learning with Cost Regularization for Multiview Stereo

MVSNet: Depth Inference for Unstructured Multi-view Stereo

Semi-supervised Deep Multi-view Stereo

Mono‐MVS: textureless‐aware multi‐view stereo assisted by monocular prediction

Self-supervised multi-frame depth estimation with visual-inertial pose transformer and monocular guidance

A Light Multi-View Stereo Method with Patch-Uncertainty Awareness

Attention Aware Cost Volume Pyramid Based Multi-view Stereo Network for 3D Reconstruction

HC-MVSNet: A Probability Sampling-Based Multi-View-stereo Network with Hybrid Cascade Structure for 3D Reconstruction

EPP-MVSNet: Epipolar-assembling based Depth Prediction for Multi-view Stereo

Stereo Matching by Self-supervision of Multiscopic Vision.

Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency

A contrastive learning based unsupervised multi-view stereo with multi-stage self-training strategy