Abstract:Consumer-level RGB-D cameras have been widely used for dense 3D reconstruction of scenes. Especially for textureless or non-lambertian surfaces, consumer RGB-D cameras can ensure completeness of the reconstructed models at a low cost. However, the reconstruction quality relies heavily on the accuracy of the depth sensors. Digital cameras are also used popularly for capturing high-resolution pictures to achieve high-quality dense reconstruction of the scenes, but cannot handle textureless or non-lambertian regions well due to the visual ambiguity problem. To ensure both completeness and accuracy of the reconstructed 3D models, we propose a hybrid multi-view reconstruction pipeline named Hybrid-MVS, which combines the high-resolution images taken by a digital camera and the low-resolution RGB-D frames captured by a consumer RGB-D camera for robust reconstruction of complicated scenes with challenging textureless and non-lambertian surfaces. Unlike most existing multi-sensor systems which require explicit hardware calibration and synchronization of various sensors, the calibration and synchronization problems between the digital camera and RGB-D camera are implicitly solved for compositing reliable depth prior of the digital images in our pipeline. Especially, we propose a hybrid MVS framework for robust PatchMatch stereo and Delaunay meshing, which tightly couples both visual cues given by the digital images and depth cues from the RGB-D frames to maximize the complementary advantages. The experiments with quantitative and qualitative evaluations demonstrate the effectiveness of the proposed Hybrid-MVS framework, which can successfully achieve high-quality 3D reconstruction of complicated natural scenes with robustness to weakly textured and non-lambertian areas.

Improving RealSense by Fusing Color Stereo Vision and Infrared Stereo Vision for the Visually Impaired

IR Stereo RealSense: Decreasing Minimum Range of Navigational Assistance for Visually Impaired Individuals.

Long-Range Traversability Awareness and Low-Lying Obstacle Negotiation with RealSense for the Visually Impaired

Expanding the Detection of Traversable Area with RealSense for the Visually Impaired

Glass Detection and Recognition Based on the Fusion of Ultrasonic Sensor and RGB-D Sensor for the Visually Impaired

Reducing the minimum range of a RGB-depth sensor to aid navigation in visually impaired individuals

Detecting Traversable Area and Water Hazards for the Visually Impaired with a Prgb-D Sensor.

Assisting the Visually Impaired: Multitarget Warning Through Millimeter Wave Radar and RGB-depth Sensors.

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

Robustifying Semantic Cognition of Traversability Across Wearable RGB-depth Cameras

An Environmental Perception and Navigational Assistance System for Visually Impaired Persons Based on Semantic Stixels and Sound Interaction

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

Real-time Depth Extraction and Multi-View Rendering Algorithm Based on Kinect

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Fusion of millimeter wave radar and RGB-depth sensors for assisted navigation of the visually impaired

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

Depth incorporating with color improves salient object detection

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

High-Resolution Depth Maps Based on TOF-Stereo Fusion