Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Modeling Long-Range Dependencies and Epipolar Geometry for Multi-View Stereo

MVSTER: Epipolar Transformer for Efficient Multi-View Stereo

Multi-View Stereo Representation Revist: Region-Aware MVSNet

MTD-MVSNet: Multi-view Stereo Network with Multi-scale Transformer and Dual Attention

A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding

Feature‐enhanced representation with transformers for multi‐view stereo

When Epipolar Constraint Meets Non-local Operators in Multi-View Stereo

Unsupervised multi-view stereo network based on multi-stage depth estimation

EPP-MVSNet: Epipolar-assembling based Depth Prediction for Multi-view Stereo

Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks

EI-MVSNet: Epipolar-Guided Multi-View Stereo Network With Interval-Aware Label

MVSNet: Depth Inference for Unstructured Multi-view Stereo

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Multi-View Stereo Network with attention thin volume

Self-supervised Multi-view Stereo Via Inter and Intra Network Pseudo Depth

GeoMVSNet: Learning Multi-View Stereo with Geometry Perception

Transformer-guided Feature Pyramid Network for Multi-View Stereo

MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo

Context-Guided Multi-view Stereo with Depth Back-Projection

CT-MVSNet: Curvature-guided multi-view stereo with transformers

CostFormer:Cost Transformer for Cost Aggregation in Multi-view Stereo