Abstract:It is widely believed that sparse supervision is worse than dense supervision in the field of depth completion, but the underlying reasons for this are rarely discussed. To this end, we revisit the task of radar-camera depth completion and present a new method with sparse LiDAR supervision to outperform previous dense LiDAR supervision methods in both accuracy and speed. Specifically, when trained by sparse LiDAR supervision, depth completion models usually output depth maps containing significant stripe-like artifacts. We find that such a phenomenon is caused by the implicitly learned positional distribution pattern from sparse LiDAR supervision, termed as LiDAR Distribution Leakage (LDL) in this paper. Based on such understanding, we present a novel Disruption-Compensation radar-camera depth completion framework to address this issue. The Disruption part aims to deliberately disrupt the learning of LiDAR distribution from sparse supervision, while the Compensation part aims to leverage 3D spatial and 2D semantic information to compensate for the information loss of previous disruptions. Extensive experimental results demonstrate that by reducing the impact of LDL, our framework with sparse supervision outperforms the state-of-the-art dense supervision methods with 11.6% improvement in Mean Absolute Error (MAE)} and 1.6x speedup in Frame Per Second (FPS)}. The code is available at <a class="link-external link-https" href="https://github.com/megvii-research/Sparse-Beats-Dense" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper primarily addresses the issue of sparse supervision in the radar-camera depth completion task. Specifically: 1. **The problem of sparse supervision**: - In depth completion tasks, it is generally believed that sparse supervision is less effective than dense supervision. However, the paper points out that when training with sparse LiDAR data, the depth maps output by the model exhibit noticeable stripe-like artifacts, making the depth maps unsuitable for downstream applications. 2. **New concept introduced**: - The paper introduces a new concept—LiDAR Distribution Leakage (LDL), which explains why sparse LiDAR supervision leads to unusable depth maps. Specifically, this phenomenon occurs because the model implicitly learns the positional distribution patterns from the sparse LiDAR data. 3. **Proposed solution**: - To address this issue, the paper proposes a new framework—the Disruption-Compensation radar-camera depth completion framework. This framework consists of two parts: - Disruption: Mitigates the impact of LDL by disrupting the positional distribution patterns of the LiDAR data. - Compensation: Utilizes 3D spatial and 2D semantic information to compensate for the information lost during the disruption process. 4. **Experimental results**: - Experimental results show that this framework significantly improves the accuracy of depth completion under sparse supervision and is faster. Specifically, compared to existing dense supervision methods, this framework improves the Mean Absolute Error (MAE) by 11.6% and increases the Frame Per Second (FPS) by 1.6 times. Through these improvements, the paper successfully addresses the depth completion problem under sparse LiDAR supervision and demonstrates its superior performance in practical applications.

Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completion

Semantic-guided Depth Completion from Monocular Images and 4D Radar Data

LODM: Large-scale Online Dense Mapping for UAV

Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera

Self-supervised Depth Completion with Attention-Based Loss

Radar-Camera Pixel Depth Association for Depth Completion

DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network

Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints

Depth Estimation via Sparse Radar Prior and Driving Scene Semantics.

Sparse Pseudo-LiDAR Depth Assisted Monocular Depth Estimation

Deep Depth Completion from Extremely Sparse Data: A Survey

RigNet++: Semantic Assisted Repetitive Image Guided Network for Depth Completion

RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scale

Expanding Sparse LiDAR Depth and Guiding Stereo Matching for Robust Dense Depth Estimation

Towards Better Unguided Depth Completion via Cross-Modality Knowledge Distillation in the Frequency Domain

SparseFusion3D: Sparse Sensor Fusion for 3D object detection by Radar and Camera in Environmental Perception

Dense Depth Completion Based on Piecewise Planar Model

Self-Supervised Depth Completion Via Adaptive Sampling and Relative Consistency

Self‐supervised Depth Completion with Multi‐view Geometric Constraints

LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Camera

Robust Depth Completion with Uncertainty-Driven Loss Functions