Abstract:Multiple-view stereo has potential applications in robotic operations and autonomous driving (unstructured environment construction, visual servo). With assisted depth information, inertial navigation systems can achieve precise navigation. It is, especially suitable for GPS failures in complex environments. Accurate depth estimation is a challenge in low-textured or occluded regions. To alleviate the inference of incorrect depth, a multi-stage pixel-visibility learning-based stereo network is presented in this paper. Its improvements are as follows: 1) a new content-adaptive cost volume aggregation mechanism based on neighboring pixel-wise visibility is designed to effectively produce more accurate and smoother depth map predictions in the object boundary. 2) global convolution block and boundary refinement block are developed to regularize its cost volume, they can learn the inherent constraints of feature matching correspondence and effectively mitigate the depth estimation uncertainty in low-textured regions. 3) a new loss function is designed to measure the uncertainty of predicted probability distribution and enhance the reliability of depth map inference. Experimental results on the indoor DTU datasets and the outdoor Tanks & Temples datasets indicate that our method can achieve superior performance and has a powerful generalization ability, which is comparable to state-of-the-art works. Note to Practitioners—Multiple-view stereo (MVS) can estimate dense 3D representations of scenes, which is widely used in autonomous driving, robotic navigation, virtual reality (VR), and augmented reality (AR). Aiming at the problem of incorrect depth inference in low-textured or occluded regions, this work proposes a novel multi-stage depth prediction method based on neighboring pixel-wise visibility. Our method cannot only achieve accurate depth estimation for robot perception but also make no concession to real-time performance. It is clear that the proposed method has good potential in 3D reconstruction, robotic navigation, and VR/AR fields to provide accurate depth estimation in real-time with limited memory consumption.

PQMPV: Parallel and Quantified Dense Stereo Disparity Estimation Based on Multi-Path Viterbi

Real-time Stereo Vision System Using Adaptive Weight Cost Aggregation Approach

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Real-time Dense Disparity Estimation Based on Multi-Path Viterbi for Intelligent Vehicle Applications.

Multi-Stage Feature Pyramid Stereo Network-Based Disparity Estimation Approach for Two to Three-Dimensional Video Conversion

Robust Depth Estimation Based on Parallax Attention for Aerial Scene Perception

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

Hybrid stereo matching by dynamic programming with enhanced cost entry for real-time depth generation

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

DPS-Net: Deep Polarimetric Stereo Depth Estimation

Disparity Estimation Using a Quad-Pixel Sensor

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

Stereo Matching Method with Integrated Geometric Encoding for Disparity Refinement

High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming.

Disparity Estimation Based on the Combination of Depth Camera and Stereo Vision

Multistage Pixel-Visibility Learning with Cost Regularization for Multiview Stereo

Real-Time Accurate Stereo Matching Using Modified Two-Pass Aggregation and Winner-Take-All Guided Dynamic Programming

Miper-MVS: Multi-scale Iterative Probability Estimation with Refinement for Efficient Multi-View Stereo

Robust and accurate depth estimation by fusing LiDAR and Stereo

Stereo Matching Accelerator With Re-Computation Scheme and Data-Reused Pipeline for Autonomous Vehicles

Fast stereo matching using adaptive guided filtering