Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Noisy Depth Maps Fusion for Multiview Stereo Via Matrix Completion

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

Sparse LIDAR Measurement Fusion with Joint Updating Cost for Fast Stereo Matching

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Select-and-Combine (SAC): A Novel Multi-Stereo Depth Fusion Algorithm for Point Cloud Generation via Efficient Local Markov Netlets

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

A matrix completion-based multiview learning method for imputing missing values in buoy monitoring data

Depth map enhancement based on color and depth consistency

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

FusionDepth: Complement Self-Supervised Monocular Depth Estimation with Cost Volume

Multiview Point Clouds Denoising Based on Interference Elimination.

End-to-end multiview fusion for building mapping from aerial images

A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video

End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds

Point Cloud Densification Algorithm for Multiple Cameras and Lidars Data Fusion

High completeness multi-view stereo for dense reconstruction of large-scale urban scenes

Depth-Guided Robust and Fast Point Cloud Fusion NeRF for Sparse Input Views

Prioritized multi-view stereo depth map generation using confidence prediction

Depth Completion via Inductive Fusion of Planar LIDAR and Monocular Camera

LSMD-Net: LiDAR-Stereo Fusion with Mixture Density Network for Depth Sensing