Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review
Yaodong Cui,Ren Chen,Wenbo Chu,Long Chen,Daxin Tian,Ying Li,Dongpu Cao
DOI: https://doi.org/10.1109/tits.2020.3023541
IF: 8.5
2022-02-01
IEEE Transactions on Intelligent Transportation Systems
Abstract:Autonomous vehicles were experiencing rapid development in the past few years. However, achieving full autonomy is not a trivial task, due to the nature of the complex and dynamic driving environment. Therefore, autonomous vehicles are equipped with a suite of different sensors to ensure robust, accurate environmental perception. In particular, the camera-LiDAR fusion is becoming an emerging research theme. However, so far there has been no critical review that focuses on deep-learning-based camera-LiDAR fusion methods. To bridge this gap and motivate future research, this article devotes to review recent deep-learning-based data fusion approaches that leverage both image and point cloud. This review gives a brief overview of deep learning on image and point cloud data processing. Followed by in-depth reviews of camera-LiDAR fusion methods in depth completion, object detection, semantic segmentation, tracking and online cross-sensor calibration, which are organized based on their respective fusion levels. Furthermore, we compare these methods on publicly available datasets. Finally, we identified gaps and over-looked challenges between current academic researches and real-world applications. Based on these observations, we provide our insights and point out promising research directions.
engineering, electrical & electronic,transportation science & technology, civil