Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Vid2Curve: Simultaneously Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video

Vid2Curve: simultaneous camera motion estimation and thin structure reconstruction from an RGB video

Vid2Curve

A Simple 3D Reconstruction Method Based on Single Camera

A Multiscale-Contour-based Interpolation Framework for Generating a Time-Varying Quasi-Dense Point Cloud Sequence.

Robust 3D Reconstruction with an RGB-D Camera

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

Rendering-oriented Multiview Video Coding Based on Chrominance Information Reconstruction

A Temporally Streamlined Optimization Method for Stereo Video Correspondence

UnstructuredFusion: Realtime 4D Geometry and Texture Reconstruction Using Commercial RGBD Cameras.

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Video Vectorization via Bipartite Diffusion Curves Propagation and Optimization

Recovering 3D Planar Arrangements from Videos

Shape of Motion: 4D Reconstruction from a Single Video

Templateless Non-Rigid Reconstruction and Motion Tracking with a Single RGB-D Camera.

Real Time Image Sequence Segmentation Using Curve Evolution

Video Motion Capture by Silhouette Analysis and Pose Optimization

Video-Based Outdoor Human Reconstruction.

Robust 3D Reconstruction from Uncalibrated Small Motion Clips

An efficient 2D to 3D video conversion method based on skeleton line tracking

Visualizing and Quantifying Small and Nonstationary Structural Motions in Video Measurement.