Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

A binocular stereo-view 3D reconstruction algorithm based on contour extraction and depth screening

A Three-Dimensional Measurement Method for Binocular Endoscopes Based on Deep Learning

Terrain Reconstruction of Lunar Surface Based on Binocular Fisheye Camera

Denoising and hole-filling for binocular stereo reconstruction

Research progress of depth detection in vision measurement: A novel project of bifocal imaging system for 3D measurement

3D reconstruction from bifocus imaging

A Three-Dimensional Reconstruction Algorithm for Complex Background Image and Its Medical Applications

Stereoscopic video conversion based on depth tracking

An improved 3D face reconstruction method

Multi-View Depth Map Sampling for 3D Reconstruction of Natural Scene

High-Quality Depth Recovery Via Interactive Multi-view Stereo

An automatic 2D to 3D conversion algorithm using multi-depth cues

Stereo Image Rectification Algorithm for Multi-View 3D Display

A Monocular Stereo Vision Algorithm Based on Bifocal Imaging

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

3d Stereoscopic Image Generation From Monocular View In Structured Environments

Robust 3D Reconstruction with an RGB-D Camera

Live Three-Dimensional Reconstruction System with Stereo Vision Display

3D Reconstruction of the Retina Based on Un-Calibrated Fundus Images

High-Precision Measurement of Binocular Telecentric Vision System With Novel Calibration and Matching Methods

Three-dimensional reconstruction optimization of tunnel face and intelligent extraction of discontinuity orientation based on binocular stereo vision