Abstract:Given a set of calibrated images, Multiple View Stereo (MVS) applies end-to-end depth inference network to recover scene structure. However, previous methods designed pixel-visibility modules to aggregate cross-view cost, ignoring the consistency assumption of 2D contextual features in the 3D depth direction. The current multi-stage depth inference model also relies on intensive depth samples, which requires high memory consumption. To alleviate these problems, this work exploits edge-assisted epipolar Transformer for multi-view depth inference. The improvements of this work are summarized as follows: 1) The epipolar Transformer block is developed for reliable cross-view cost aggregation, and the edge detection branch is designed to constrain the consistency of epipolar geometry and edge features. 2) The dynamic depth range sampling mechanism based on probability volume is adopted to improve the accuracy of uncertain areas. Comprehensive comparisons with the state-of-the-art works indicate that our work can reconstruct dense scene representations with limited memory bottleblock Note to Practitioners —Learning-based MVS can obtain dense point clouds with accurate depth map estimation, which are widely applied in the fields of unmanned driving, battlefield environment perception and robot navigation. MVS-based scene reconstruction technology is the premise of the subsequent planning, decision-making and control of the human-machine system. To obtain dense scene representation with limited memory and runtime, this work proposes a multi-view stereo network with edge-assisted epipolar Transformer. Experiments on public benchmarks verify the feasibility and effectiveness of our model, which has good potential in battlefield environment reconstruction and human-computer interaction fields, and can provide intuitive and dense scene representation for decision-making assistance.

Better Together: Shading Cues and Multi-View Stereo for Reconstruction Depth Optimization

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

Multi-View Stereo Representation Revist: Region-Aware MVSNet

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

Robust, Efficient Depth Reconstruction with Hierarchical Confidence-Based Matching.

Outdoor Scene 3D Reconstruction from Multiple Point Cloud

Semantic Reconstruction based on RGB Image and Sparse Depth

A Light Multi-View Stereo Method with Patch-Uncertainty Awareness

Adaptive aggregation and depth refinement multi-view stereo network

Recurrent Multi-view Stereo Depth Inference with Pyramid of Images

Improving completeness and accuracy of 3D point clouds by using deep learning for applications of digital twins to civil structures

3D Reconstruction of Typical Entities Based on Multi-Perspective Images

Depth Estimation in Multi-View Stereo Based on Image Pyramid

Enhanced multi view 3D reconstruction with improved MVSNet

Multi-View Stereo Via Geometric Expansion And Depth Refinement

High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF

Edge-Assisted Epipolar Transformer for Industrial Scene Reconstruction

3d Reconstruction Of Dynamic Scenes With Multiple Handheld Cameras