Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Multi-view 3D Morphable Face Reconstruction via Canonical Volume Fusion

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Recurrent Volume-Based 3-D Feature Fusion for Real-Time Multiview Object Pose Estimation.

A Lightweight Network with Multi-Stage Feature Fusion Module for Single-View 3d Face Reconstruction.

From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion

MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

3d Reconstruction With Multi-View Texture Mapping

Disentangling 3D/4D Facial Affect Recognition with Faster Multi-View Transformer

Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape

Multi-view 3D Reconstruction with Transformer

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Self-Supervised 3d Face Reconstruction Based On Multi-View Uv Fusion

AdaptiveFusion: Adaptive Multi-Modal Multi-View Fusion for 3D Human Body Reconstruction

Multi-View Human Mesh Reconstruction via Direction-Aware Feature Fusion

VPFusion: Joint 3D Volume and Pixel-Aligned Feature Fusion for Single and Multi-view 3D Reconstruction

A Lightweight Monocular 3D Face Reconstruction Method Based on Improved 3D Morphing Models.

High-Quality 3D Face Reconstruction with Affine Convolutional Networks

View Consistent 3D Face Reconstruction Using Siamese Encoder-Decoders

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.