Abstract:The combination of range sensors with color cameras can be very useful for robot navigation, semantic perception, manipulation, and telepresence. Several methods of combining range- and color-data have been investigated and successfully used in various robotic applications. Most of these systems suffer from the problems of noise in the range-data and resolution mismatch between the range sensor and the color cameras, since the resolution of current range sensors is much less than the resolution of color cameras. High-resolution depth maps can be obtained using stereo matching, but this often fails to construct accurate depth maps of weakly/repetitively textured scenes, or if the scene exhibits complex self-occlusions. Range sensors provide coarse depth information regardless of presence/absence of texture. The use of a calibrated system, composed of a time-of-flight (TOF) camera and of a stereoscopic camera pair, allows data fusion thus overcoming the weaknesses of both individual sensors. We propose a novel TOF-stereo fusion method based on an efficient seed-growing algorithm which uses the TOF data projected onto the stereo image pair as an initial set of correspondences. These initial "seeds" are then propagated based on a Bayesian model which combines an image similarity score with rough depth priors computed from the low-resolution range data. The overall result is a dense and accurate depth map at the resolution of the color cameras at hand. We show that the proposed algorithm outperforms 2D image-based stereo algorithms and that the results are of higher resolution than off-the-shelf color-range sensors, e.g., Kinect. Moreover, the algorithm potentially exhibits real-time performance on a single CPU.

Real-time high resolution 2D-plus-depth video generation using time-of-flight sensor

DoCam: Depth Sensing with an Optical Image Stabilization Supported RGB Camera.

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

High quality depth maps from stereo matching and ToF camera.

Hardware Solution of Real-Time Depth Estimation Based on Stereo Vision

Full High-Definition Real-Time Depth Estimation for Three-Dimensional Video System

Full Hd Real-Time Depth Estimation Algorithm and Hardware Implementation

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Fast frame-rate up-conversion of depth video via video coding.

TOF Depth Map Super-resolution Using Compressive Sensing

High-Resolution Depth Maps Based on TOF-Stereo Fusion

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

Depth Map Super-Resolution and Enhancement for Time-Of-Flight Cameras

Temporal Upsampling of Depth Maps Using a Hybrid Camera

Depth super resolution using bilateral filter

Depth Upsampling Method Via Markov Random Fields Without Edge-Misaligned Artifacts

Robust High Quality Image Guided Depth Upsampling

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

Real-time Depth Extraction and Multi-View Rendering Algorithm Based on Kinect

A Real-Time Full HD 2D-to-3d Video Conversion System Based on FPGA

An automatic 2D to 3D conversion algorithm using multi-depth cues