Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Gradient-domain-based Enhancement of Multi-View Depth Video

A Comparative Study on Guided Depth Enhancement

Improved Multi-View Depth Estimation For View Synthesis In 3d Video Coding

Applications of just-noticeable depth difference model in joint multiview video plus depth coding

A Depth Extraction Method Based on Motion and Geometry for 2D to 3D Conversion

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Rendering-oriented Multiview Video Coding Based on Chrominance Information Reconstruction

An automatic 2D to 3D conversion algorithm using multi-depth cues

A novel depth coding scheme for multiview video plus depth compression

A Temporally Streamlined Optimization Method for Stereo Video Correspondence

Reduced Resolution Depth Compression for Multiview Video Plus Depth Coding

Multi-view Image Depth Estimation for Coding and View Rendering Synthesis

Temporally Consistent Depth Map Estimation Based On 3d-Mrf

Low Complexity Multiview Video Plus Depth Coding

Multiview Video Depth Estimation with Spatial-Temporal Consistency

Geometric mapping assisted multi-view depth video coding

Asymmetric Coding of Multi-View Video Plus Depth Based 3-D Video for View Rendering

A Multi-View Video Coding Approach Using Layered Depth Image

Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning

A Multi-View Fusion Method Via Tensor Learning And Gradient Descent For Image Features

Depth Super-Resolution Via Joint Color-Guided Internal and External Regularizations