Abstract:Depth completion aims to recover pixelwise depth from incomplete and noisy depth measurements with or without the guidance of a reference RGB image. This task attracted considerable research interest due to its importance in various computer vision-based applications, such as scene understanding, autonomous driving, 3-D reconstruction, object detection, pose estimation, trajectory prediction, and so on. As the system input, an incomplete depth map is usually generated by projecting the 3-D points collected by ranging sensors, such as LiDAR in outdoor environments, or obtained directly from RGB-D cameras in indoor areas. However, even if a high-end LiDAR is employed, the obtained depth maps are still very sparse and noisy, especially in the regions near the object boundaries, which makes the depth completion task a challenging problem. To address this issue, a few years ago, conventional image processing-based techniques were employed to fill the holes and remove the noise from the relatively dense depth maps obtained by RGB-D cameras, while deep learning-based methods have recently become increasingly popular and inspiring results have been achieved, especially for the challenging situation of LiDAR-image-based depth completion. This article systematically reviews and summarizes the works related to the topic of depth completion in terms of input modalities, data fusion strategies, loss functions, and experimental settings, especially for the key techniques proposed in deep learning-based multiple input methods. On this basis, we conclude by presenting the current status of depth completion and discussing several prospects for its future research directions.

Self-Supervised Single-Line LiDAR Depth Completion

Semantic-guided Depth Completion from Monocular Images and 4D Radar Data

Least Square Estimation Network for Depth Completion

Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

Lambertian Model-Based Normal Guided Depth Completion for LiDAR-Camera System

SGTBN: Generating Dense Depth Maps from Single-Line LiDAR

Self-Supervised Depth Completion Via Adaptive Sampling and Relative Consistency

Spatiotemporal Guided Self-Supervised Depth Completion from LiDAR and Monocular Camera

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

Self-supervised Depth Completion with Attention-Based Loss

Deterministic Guided LiDAR Depth Map Completion

Self-Supervised Depth Completion Guided by 3D Perception and Geometry Consistency

Self‐supervised Depth Completion with Multi‐view Geometric Constraints

Self-supervised Multi-frame Depth Completion Based on Local-depth-field Cost Volume

Depth Completion with Edge Optimization for Depth Maps from Lidar

Recent Advances in Conventional and Deep Learning-Based Depth Completion: A Survey

RigNet++: Semantic Assisted Repetitive Image Guided Network for Depth Completion

Deep Depth Completion from Extremely Sparse Data: A Survey

Self-Supervised Depth Completion From Direct Visual-LiDAR Odometry in Autonomous Driving