Snow-CLOCs: Camera-LiDAR Object Candidate Fusion for 3D Object Detection in Snowy Conditions

Xiangsuo Fan,Dachuan Xiao,Qi Li,Rui Gong
DOI: https://doi.org/10.3390/s24134158
IF: 3.9
2024-06-27
Sensors
Abstract:Although existing 3D object-detection methods have achieved promising results on conventional datasets, it is still challenging to detect objects in data collected under adverse weather conditions. Data distortion from LiDAR and cameras in such conditions leads to poor performance of traditional single-sensor detection methods. Multi-modal data-fusion methods struggle with data distortion and low alignment accuracy, making accurate target detection difficult. To address this, we propose a multi-modal object-detection algorithm, Snow-CLOCs, specifically for snowy conditions. In image detection, we improved the YOLOv5 algorithm by integrating the InceptionNeXt network to enhance feature extraction and using the Wise-IoU algorithm to reduce dependency on high-quality data. For LiDAR point-cloud detection, we built upon the SECOND algorithm and employed the DROR filter to remove noise, enhancing detection accuracy. We combined the detection results from the camera and LiDAR into a unified detection set, represented using a sparse tensor, and extracted features through a 2D convolutional neural network to achieve object detection and localization. Snow-CLOCs achieved a detection accuracy of 86.61% for vehicle detection in snowy conditions.
engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
What problem does this paper attempt to address?
The paper primarily addresses the challenges of 3D object detection faced by autonomous vehicles under snowy conditions. Specifically, the paper tackles the following issues: 1. **Image Data Distortion**: In snowy environments, images captured by cameras are affected by snowflakes, leading to a decline in image quality, which in turn affects the accuracy of vision-based object detection algorithms. 2. **Point Cloud Data Noise**: LiDAR data collected under snowy conditions contains a significant amount of noise generated by snowflakes, reducing the quality of the point cloud data and further impacting the performance of LiDAR-based object detection algorithms. To address these challenges, the authors propose a multimodal object detection algorithm named Snow-CLOCs (Camera-LiDAR Object Candidate Fusion). The algorithm includes the following innovations: - **Image Processing Improvement**: The DTCWT (Dual-Tree Complex Wavelet Transform) snow removal algorithm is used to enhance image quality and reduce the impact of snowflakes on images. - **YOLOv5 Enhancement**: By integrating the InceptionNeXt network into the YOLOv5 framework to improve feature extraction capabilities and using the Wise-IoU algorithm to reduce dependency on high-quality datasets, the accuracy of image object detection is improved. - **LiDAR Point Cloud Filtering**: The DROR (Dynamic Radius Outlier Removal) filtering algorithm is introduced to reduce noise in LiDAR point clouds, enhancing the accuracy of point cloud object detection. - **Backend Fusion Method**: The detection results from images and LiDAR are combined using geometric consistency and semantic consistency to achieve more accurate object detection and localization. Through these improvements, the Snow-CLOCs algorithm achieved a vehicle detection accuracy of 86.61% under snowy conditions. This achievement is significant for enhancing the perception capabilities and safety of autonomous vehicles in adverse weather conditions.