Abstract:We propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) They assume the given sparse depth map is accurately aligned to the input image, whereas the alignment is difficult to achieve in practice. (ii) They have limited accuracy in the long range because the depth is estimated by pixel disparity. To solve the abovementioned limitations, we propose selective stereo matching (SSM) that searches the most appropriate depth value for each image pixel from its neighborly projected LiDAR points based on an energy minimization framework. This depth selection approach can handle any type of mis-projection. Moreover, SSM has an advantage in terms of long-range depth accuracy because it directly uses the LiDAR measurement rather than the depth acquired from the stereo. SSM is a discrete process; thus, we apply variational smoothing with binary anisotropic diffusion tensor (B-ADT) to generate a continuous depth map while preserving depth discontinuity across object boundaries. Experimentally, compared with the previous state-of-the-art stereo-aided depth completion, the proposed method reduced the mean absolute error (MAE) of the depth estimation to 0.65 times and demonstrated approximately twice more accurate estimation in the long range. Moreover, under various LiDAR-camera calibration errors, the proposed method reduced the depth estimation MAE to 0.34-0.93 times from previous depth completion methods.

Learning Sparse Representations of Depth

Prior surface model guided dense reconstruction via graph-cuts

Depth Estimation from Single Image using Sparse Representations

Connecting The Dots: Learning Representations For Active Monocular Depth Estimation

Reconstructing Sparsely Represented High-resolution Depth Map from Corrupted Samples

Learning joint intensity-depth sparse representations

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

Estimating Depth from RGB and Sparse Sensing

Sparse LIDAR Measurement Fusion with Joint Updating Cost for Fast Stereo Matching

Sparse-to-Dense Depth Estimation in Videos Via High-Dimensional Tensor Voting

ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images

Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching

Deep Depth Estimation from Visual-Inertial SLAM

A Bayesian framework for dense depth estimation based on spatial-temporal correlation

Learned Semantic Multi-Sensor Depth Map Fusion

Depth Refinement for Improved Stereo Reconstruction

Just-in-Time Reconstruction: Inpainting Sparse Maps using Single View Depth Predictors as Priors

DELTAS: Depth Estimation by Learning Triangulation And densification of Sparse points

Sparse-to-Continuous: Enhancing Monocular Depth Estimation using Occupancy Maps

Fusion of Sparse LIDAR and Uncertainty-Based 3D Vision for Accurate Depth Estimation with Bayesian Kriging

Learning a Depth Covariance Function