Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Hybrid Method for Multiple Viewpoint Projection Holography

Multiplane Digital Holography Based on Extrapolation Iterations

Holographic near-eye display system based on double-convergence light Gerchberg-Saxton algorithm.

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Multiview and light-field reconstruction algorithms for 360° multiple-projector-type 3D display.

A Novel Method of Multi-view Virtual Image Synthesis for Auto-stereoscopic Display

Image mosaic based view synthesis for interactive stereoscopic display

MVP-Stereo: A Parallel Multi-View Patchmatch Stereo Method with Dilation Matching for Photogrammetric Application

Stereo Unstructured Magnification: Multiple Homography Image for View Synthesis

Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators

Research on Digital Synthetic Holography Based on Multi-Parallax Stereo Display

Dual-View Integral Imaging 3D Display Based on Multiplexed Lens-Array Holographic Optical Element

Lensless holographic dynamic projection system based on weakly supervised learning

Volume Holographic Printing Using Unconventional Angular Multiplexing for Three-Dimensional Display

A generalized iterative projection framework for pixel-super-resolved holographic imaging

A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video

Reconstructed Quality Improvement with a Stochastic Gradient Descent Optimization Algorithm for a Spherical Hologram.

Michelson Holography: Dual-SLM Holography with Camera-in-the-loop Optimization

Three-dimensional Hologram Generation Method Based on Space-Division

Layered Holographic Stereogram Based on Inverse Fresnel Diffraction

Vision Transformer-Based, High-Fidelity, Computer-Generated Holography