Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Object Segmentation Ensuring Consistency Across Multi-Viewpoint Images

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Multi-objects Real Time Recognition Based on Color Information

Simultaneous multi-body stereo and segmentation

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Joint Multiview Segmentation And Localization Of Rgb-D Images Using Depth-Induced Silhouette Consistency

Object segmentation from sparse views of wide-baseline images.

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Multi-view Object Extraction with Fractional Boundaries

Multi-view foreground segmentation based on spatial consistency constraints

Joint Object Segmentation and Depth Upsampling

Object segmentation based on disparity estimation

Automatic Object Cosegmentation In Sparse Multiview Images

Contour Based Automatic Scene Segmentation in Image Sequences

Object Segmentation in Multiple Views Without Camera Calibration

Object Detection and Tracking under Occlusion for Object-Level RGB-D Video Segmentation

Multi-View Video Based Multiple Objects Segmentation Using Graph Cut and Spatiotemporal Projections

Interactive Object Segmentation from Multi-View Images.

Object Proposal via Depth Connectivity Constrained Grouping.