Abstract:Multi-modal fusion is imperative to the implementation of reliable autonomous driving. In this paper, we make full use of both mmWave radar and camera data to reconstruct reliable depth and full-velocity information. To overcome the sparsity of radar points, we leverage full-velocity based egomotion compensation to achieve more accurate multi-sweep accumulation. Besides, we incorporate an adaptive two-stage attention module within the fusion network to exploit the synergy of camera and radar information. Furthermore, we conduct extensive experiments on the prevailing nuScenes dataset. The results show our proposed fusion system consistently outperforms the state-of-the-art methods.

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving