Abstract:The inherent sparsity of LiDAR data often leads to extremely sparse depth maps, which poses a challenge for the development of LiDAR-based egocentric vehicles, such as self-driving cars and mobile robots. To overcome this limitation, guided depth completion methods use calibrated camera images to create precise, dense depth predictions from sparse LiDAR data. However, the extreme reliance on camera image limits its generalization of guided depth completion, especially its robustness to weather and light. In this paper, we aim to utilize camera images only during training phase to improve unguided depth completion, and discard camera in the inference phase. We comprehensively analyze the pivotal role of camera images in the depth completion task and emphasize the significance of the frequency distribution within the local windows, quantitatively demonstrating its substantial contribution. Subsequently, we introduce cross-modality knowledge distillation to align LiDAR features with camera features in the frequency domain, yielding corresponding guidance features. We devise a guidance and selection module to mitigate unavoidable inaccuracies in knowledge distillation, while it can enhance depth features and adeptly selects more precise encoded values from both the guidance branch and the unguided input. To further refine the completion result, we propose a progressive depth completion module incorporating two sub-networks connected by an attention for refinement module. This module produces weighted features from the decoder of the first stage to enhance the features in the encoder of the second stage. We denominate our method as Better Unguided Network (BUNet) and evaluate its efficacy on the KITTI depth completion benchmark and NYUv2 dataset, demonstrating its superiority over methods that exclude camera images during the inference phase.

Spatiotemporal Guided Self-Supervised Depth Completion from LiDAR and Monocular Camera

Semantic-guided Depth Completion from Monocular Images and 4D Radar Data

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion

Self-supervised Depth Completion with Attention-Based Loss

Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera

Spatiotemporally Enhanced Photometric Loss for Self-Supervised Monocular Depth Estimation.

Self-Supervised Depth Completion From Direct Visual-LiDAR Odometry in Autonomous Driving

Towards Better Unguided Depth Completion via Cross-Modality Knowledge Distillation in the Frequency Domain

Self-Supervised Depth Completion Guided by 3D Perception and Geometry Consistency

Self‐supervised Depth Completion with Multi‐view Geometric Constraints

Self-Supervised Depth Completion Based on Multi-Modal Spatio-Temporal Consistency.

Self-Supervised Single-Line LiDAR Depth Completion

Self-supervised Multi-frame Depth Completion Based on Local-depth-field Cost Volume

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

Unsupervised Depth Completion Guided by Visual Inertial System and Confidence

Self-Supervised Monocular Depth Estimation Based on High-Order Spatial Interactions

Unsupervised Depth Completion From Visual Inertial Odometry

Real-time depth completion based on LiDAR-stereo for autonomous driving

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

Monocular Depth Estimation Using Self-Supervised Learning with More Effective Geometric Constraints