RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radar

Fangqiang Ding,Xiangyu Wen,Yunzhou Zhu,Yiming Li,Chris Xiaoxuan Lu

2024-10-28

Abstract:3D occupancy-based perception pipeline has significantly advanced autonomous driving by capturing detailed scene descriptions and demonstrating strong generalizability across various object categories and shapes. Current methods predominantly rely on LiDAR or camera inputs for 3D occupancy prediction. These methods are susceptible to adverse weather conditions, limiting the all-weather deployment of self-driving cars. To improve perception robustness, we leverage the recent advances in automotive radars and introduce a novel approach that utilizes 4D imaging radar sensors for 3D occupancy prediction. Our method, RadarOcc, circumvents the limitations of sparse radar point clouds by directly processing the 4D radar tensor, thus preserving essential scene details. RadarOcc innovatively addresses the challenges associated with the voluminous and noisy 4D radar data by employing Doppler bins descriptors, sidelobe-aware spatial sparsification, and range-wise self-attention mechanisms. To minimize the interpolation errors associated with direct coordinate transformations, we also devise a spherical-based feature encoding followed by spherical-to-Cartesian feature aggregation. We benchmark various baseline methods based on distinct modalities on the public K-Radar dataset. The results demonstrate RadarOcc's state-of-the-art performance in radar-based 3D occupancy prediction and promising results even when compared with LiDAR- or camera-based methods. Additionally, we present qualitative evidence of the superior performance of 4D radar in adverse weather conditions and explore the impact of key pipeline components through ablation studies.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning,Robotics

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to improve the 3D occupancy prediction ability of self - driving cars under various weather conditions. Specifically, the existing 3D occupancy prediction methods mainly rely on LiDAR or camera inputs, and these methods perform poorly in bad weather conditions (such as fog, rain and snow), which limits the all - weather deployment of self - driving vehicles. To solve this problem, the paper introduces a new method - RadarOcc, which uses 4D imaging radar sensors for 3D occupancy prediction.

### Main contributions:
1. **Propose RadarOcc**: This is the first 3D occupancy prediction method based on 4D radar. The paper points out that the traditional radar point cloud will lose key environmental signals during the generation process, so it advocates using 4D radar tensors (4DRT) for occupancy perception.
2. **Develop a new processing pipeline**: To deal with the challenges brought by 4DRT (such as large amount of data, much noise, and coordinate conversion problems), the paper proposes a series of techniques, including Doppler bins descriptor encoding, sidelobe - aware spatial sparsification, range self - attention mechanism, spherical - coordinate feature encoding and spherical - to - Cartesian feature aggregation.
3. **Extensive experimental verification**: The paper benchmarks RadarOcc on the K - Radar dataset and compares it with the state - of - the - art methods based on different modalities, verifying its superior performance in radar - based 3D occupancy prediction, especially its robustness in bad weather conditions.

### Technical details:
- **Data volume reduction**: Through Doppler bins descriptor encoding and sidelobe - aware spatial sparsification, reduce the data volume of 4DRT and improve the processing efficiency.
- **Spherical - coordinate feature encoding**: Directly encode spatial features in the spherical coordinate system to avoid interpolation errors caused by coordinate transformation.
- **Range self - attention mechanism**: Further reduce sidelobe interference through the range self - attention mechanism and improve the quality of feature representation.
- **Deformable self - attention**: Use 3D sparse convolution and deformable self - attention mechanism for efficient feature encoding and aggregation.
- **Spherical - to - Cartesian feature aggregation**: Aggregate spherical - coordinate features in a learnable way by defining 3D volume queries in the Cartesian coordinate system to avoid interpolation errors.

### Experimental results:
- **Quantitative evaluation**: On the K - Radar dataset, RadarOcc shows the state - of - the - art performance in the 3D occupancy prediction task, especially excellent in comparison with LiDAR and camera methods.
- **Qualitative evaluation**: Through qualitative analysis, verify the superior robustness of 4D radar data in bad weather conditions.

In conclusion, this paper significantly improves the 3D occupancy prediction ability of self - driving cars under various weather conditions by introducing the RadarOcc method, providing a new solution for achieving all - weather self - driving.