Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Simultaneous Localization and Appearance Estimation with a Consumer RGB-D Camera.

Robust 3D Reconstruction with an RGB-D Camera

Detailed Surface Geometry and Albedo Recovery from RGB-D Video under Natural Illumination

Online Global Non-rigid Registration for 3D Object Reconstruction Using Consumer-level Depth Cameras

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Real-time High-accuracy Three-Dimensional Reconstruction with Consumer RGB-D Cameras

AppFusion: Interactive Appearance Acquisition Using a Kinect Sensor

Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera

A Robust Fusion Method For Rgb-D Slam

High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF

Robust and Efficient RGB-D SLAM in Dynamic Environments

A Robust RGB-D SLAM System With Points and Lines for Low Texture Indoor Environments

6D Camera Relocalization in Visually Ambiguous Extreme Environments

Real-Time Accurate 3D Head Tracking and Pose Estimation with Consumer RGB-D Cameras

Deep Inverse Rendering for Practical Object Appearance Scan with Uncalibrated Illumination

Joint Multiview Segmentation And Localization Of Rgb-D Images Using Depth-Induced Silhouette Consistency

Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images

Real-time Indoor Scene Reconstruction with RGBD and Inertial Input.

A Self-Calibrated Photo-Geometric Depth Camera

Dynamic 3D Surface Reconstruction Using a Hand-Held Camera

Texture-less Object Detection and 6D Pose Estimation in RGB-D Images.