Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Content-based 3D Mosaics for Dynamic Urban Scenes

Content-Based Dynamic 3D Mosaics

3D Scene Modeling and Understanding from Image Sequences

3D and Moving Target Extraction from Dynamic Pushbroom Stereo Mosaics

Mosaic Based View Enlargement For Moving Objects In Moving Pictures

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Automatically Creating Stereo Parallel Mosaics for 3D Reconstructing Urban Scenes

Fast Construction of Dynamic and Multi-Resolution 360° Panoramas from Video Sequences

Fast Generation Of Dynamic And Multi-Resolution 360 Degrees Panorama From Video Sequences

Creating full view panoramic image mosaics and environment maps

An automatic 2D to 3D conversion algorithm using multi-depth cues

An Efficient Video Mosaic Panorama Algorithm

Asymmetric representation for 3D panoramic video

Robust 3D Reconstruction with an RGB-D Camera

DP-MVS: Detail Preserving Multi-View Surface Reconstruction of Large-Scale Scenes

Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment

A robust image mosaicing technique capable of creating integrated panoramas

Interactive Construction of 3D Models from Panoramic Mosaics

Cylindrical Panoramic Mosaicing from a Pipeline Video Through MRF Based Optimization.

A Robust Mosaic Panorama Technique for Video

Monocular Dense 3D Reconstruction of a Complex Dynamic Scene from Two Perspective Frames