Expanding Sparse Radar Depth Based on Joint Bilateral Filter for Radar-Guided Monocular Depth Estimation

Chen-Chou Lo,Patrick Vandewalle
DOI: https://doi.org/10.3390/s24061864
IF: 3.9
2024-03-14
Sensors
Abstract:Radar data can provide additional depth information for monocular depth estimation. It provides a cost-effective solution and is robust in various weather conditions, particularly when compared with lidar. Given the sparse and limited vertical field of view of radar signals, existing methods employ either a vertical extension of radar points or the training of a preprocessing neural network to extend sparse radar points under lidar supervision. In this work, we present a novel radar expansion technique inspired by the joint bilateral filter, tailored for radar-guided monocular depth estimation. Our approach is motivated by the synergy of spatial and range kernels within the joint bilateral filter. Unlike traditional methods that assign a weighted average of nearby pixels to the current pixel, we expand sparse radar points by calculating a confidence score based on the values of spatial and range kernels. Additionally, we propose the use of a range-aware window size for radar expansion instead of a fixed window size in the image plane. Our proposed method effectively increases the number of radar points from an average of 39 points in a raw radar frame to an average of 100 K points. Notably, the expanded radar exhibits fewer intrinsic errors when compared with raw radar and previous methodologies. To validate our approach, we assess our proposed depth estimation model on the nuScenes dataset. Comparative evaluations with existing radar-guided depth estimation models demonstrate its state-of-the-art performance.
engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
What problem does this paper attempt to address?
The main problems that this paper attempts to solve are the sparsity of radar data in monocular depth estimation and the problem of limited vertical field of view. Specifically, compared with LiDAR (Light Detection and Ranging) and RGB image data, radar signals are very sparse. There are less than 50 radar points on average in each 900×1600 image, and the density is only 0.003%. In addition, when the radar signal is projected onto the corresponding image, inaccuracy will be introduced, which further complicates the fusion of the radar and the depth - estimation model. To address these problems, the author proposes a radar extension method based on the Joint Bilateral Filter (JBF). The main contributions of this method include: 1. **Calculating the confidence map using the joint bilateral filter**: Expand sparse radar points by combining spatial and color intensity differences, thereby increasing the number of radar points. 2. **Using distance - aware window size**: Dynamically adjust the size of the expansion window according to the distance of the radar points, providing a better expansion area and higher resolution. 3. **Significantly increasing the number of radar points**: Increase the number of radar points from an average of 39 to more than 100,000 while keeping the internal error within an acceptable range. 4. **No LiDAR supervision required**: This method does not rely on LiDAR data for training and is suitable for datasets without LiDAR or unsupervised datasets. 5. **Performing excellently under multiple evaluation metrics**: Under the same model settings, this method outperforms existing radar pre - processing methods in terms of depth - estimation performance. Through these improvements, the paper aims to improve the performance of radar - guided monocular depth - estimation models, especially in scenarios with sparse data and complex environments.