Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

LoliMVS: An End-to-End Network for Multiview Stereo With Low-Light Images

LNMVSNet: A Low-Noise Multi-View Stereo Depth Inference Method for 3D Reconstruction

Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency

HC-MVSNet: A Probability Sampling-Based Multi-View-stereo Network with Hybrid Cascade Structure for 3D Reconstruction

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

NR-MVSNet: Learning Multi-View Stereo Based on Normal Consistency and Depth Refinement

DDL-MVS: Depth Discontinuity Learning for MVS Networks

Multi-View Stereo Network Based on Attention Mechanism and Neural Volume Rendering

Attention Aware Cost Volume Pyramid Based Multi-view Stereo Network for 3D Reconstruction

Vis-MVSNet: Visibility-Aware Multi-view Stereo Network

A Multitask Network for Multiview Stereo Reconstruction: When Semantic Consistency-Based Clustering Meets Depth Estimation Optimization

Learning-based Multi-View Stereo: A Survey

Multi-View Stereo Representation Revist: Region-Aware MVSNet

Blendedmvs: A Large-Scale Dataset For Generalized Multi-View Stereo Networks

SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsis

Mono‐MVS: textureless‐aware multi‐view stereo assisted by monocular prediction

DSC-MVSNet: attention aware cost volume regularization based on depthwise separable convolution for multi-view stereo

SA-MVSNet: Self-attention-based multi-view stereo network for 3D reconstruction of images with weak texture

RC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering

RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo

OD-MVSNet: Omni-dimensional dynamic multi-view stereo network