Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

A Multi-Camera Approach to Image-Based Rendering and 3-D/multiview Display of Ancient Chinese Artifacts

Realistic and interactive image-based rendering of ancient chinese artifacts using a multiple camera array

Immersive Display and Interactive Techniques for the Modeling and Rendering of Virtual Heritage Environments

Modeling and immersive display of cultural heritage

An Ultra Large Area Scanner for Ancient Painting and Calligraphy

Image-based Compression, Prioritized Transmission and Progressive Rendering of Circular Light Fields (CLFS) for Ancient Chinese Artifacts

Three Dimensional Reconstruction Using a Lenslet Light Field Camera

Image Based Modeling And Rendering For Dunhuang Art Cave

Immersive Virtual Tour Using Multi-projection with Remote Cluster Rendering

Multiview and light-field reconstruction algorithms for 360° multiple-projector-type 3D display.

Digital Modeling and Display of Ancient Architecture Based on Multi-Station Laser Scanning

A METHOD OF 3D MEASUREMENT AND RECONSTRUCTION FOR CULTURAL RELICS IN MUSEUMS

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

360-degree large-scale multiprojection light-field 3D display system.

A Novel Image-Based Rendering System With A Longitudinally Aligned Camera Array

Real 3D Digital Method for Large-Scale Cultural Heritage Sites

Development of a large-scale 3D digitizing system for cultural heritage documentation

Dynamic Panoramic Image Synthesis Technology Using Compound Eye Camera Array(Invited)

3D Holoscopic Imaging for Cultural Heritage Digitalisation

Real-Time Lens Based Rendering Algorithm for Super-Multiview Integral Photography Without Image Resampling

An IBR Method: Panoramic Mosaics of Slit Images with Depth