Abstract:This work addresses the challenging task of LiDAR-based 3D object detection in foggy weather. Collecting and annotating data in such a scenario is very time, labor and cost intensive. In this paper, we tackle this problem by simulating physically accurate fog into clear-weather scenes, so that the abundant existing real datasets captured in clear weather can be repurposed for our task. Our contributions are twofold: 1) We develop a physically valid fog simulation method that is applicable to any LiDAR dataset. This unleashes the acquisition of large-scale foggy training data at no extra cost. These partially synthetic data can be used to improve the robustness of several perception methods, such as 3D object detection and tracking or simultaneous localization and mapping, on real foggy data. 2) Through extensive experiments with several state-of-the-art detection approaches, we show that our fog simulation can be leveraged to significantly improve the performance for 3D object detection in the presence of fog. Thus, we are the first to provide strong 3D object detection baselines on the Seeing Through Fog dataset. Our code is available at <a class="link-external link-http" href="http://www.trace.ethz.ch/lidar_fog_simulation" rel="external noopener nofollow">this http URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges faced by LiDAR - based 3D object detection under foggy conditions. Specifically, the paper points out that collecting and annotating data in foggy days is very time - consuming, labor - intensive and costly. To overcome this problem, the authors propose a physically accurate fog simulation method, which can transform real - world scenes in clear weather into foggy - day scenes, thus obtaining large - scale foggy - day training data without additional cost. These partially synthesized data can be used to improve the robustness of multiple perception methods (such as 3D object detection, tracking or simultaneous localization and mapping) on real - world foggy - day data. ### Main contributions of the paper: 1. **Physically effective fog simulation method**: Developed a physically effective fog simulation method applicable to any LiDAR dataset. This method can generate large - scale foggy - day training data without additional cost. These partially synthesized data can be used to improve the robustness of 3D object detection, tracking or simultaneous localization and mapping on real - world foggy - day data. 2. **Experimental verification**: Through extensive experiments using multiple state - of - the - art detection methods, it is proved that their fog simulation method can significantly improve the 3D object detection performance under foggy conditions. Therefore, they are the first research team to provide a strong 3D object detection baseline on the Seeing Through Fog dataset. ### Key technical details: - **Optical system model**: Based on the optical system model of the LiDAR sensor, the authors derived the expression of the received signal power under foggy conditions. They distinguished the impulse responses of the system in clear and foggy weather and established a formal connection between them. This connection enables the direct modification of the distance and intensity of each original clear - weather point to simulate the measurement results under foggy conditions. - **Algorithm implementation**: Proposed a simple algorithm for simulating fog effects on clear - weather point clouds. This algorithm takes into account the attenuation and scattering effects of fog on the signal and calculates the response of the soft target (fog) by the numerical integration method. ### Experimental results: - **Quantitative results**: On the dense - fog test set of the STF dataset, multiple 3D object detection models (such as PV - RCNN, PointRCNN, SECOND, Part - A2 and PointPillars) trained with their proposed fog simulation method outperform the baseline models trained only on clear - weather data in terms of 3D average precision (AP) and 3D mean average precision (mAP). - **Qualitative results**: Compared with the existing fog simulation methods, their method is visually closer to the real foggy - day scenes, especially better in simulating fog noise. In conclusion, this paper solves the challenges faced by 3D object detection under foggy conditions through a physically accurate fog simulation method, providing a new solution for improving the perception ability of autonomous vehicles in bad weather conditions.

Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

LiDAR Snowfall Simulation for Robust 3D Object Detection

Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection

Towards Robust 3D Object Detection In Rainy Conditions

3D Object Detection Algorithm in Adverse Weather Conditions Based on LiDAR-Radar Fusion

Effect of Fog Particle Size Distribution on 3D Object Detection Under Adverse Weather Conditions

Performance Modeling a Near-Infrared ToF LiDAR Under Fog: A Data-Driven Approach

Robust 3D Object Detection in Cold Weather Conditions

Learn to Model and Filter Point Cloud Noise for a Near-Infrared ToF LiDAR in Adverse Weather

Impact of Multi-Scattered LiDAR Returns in Fog

A Foggy Weather Simulation Algorithm for Traffic Image Synthesis Based on Monocular Depth Estimation

TransFusion: Multi-Modal Robust Fusion for 3D Object Detection in Foggy Weather Based on Spatial Vision Transformer

Sunshine to Rainstorm: Cross-Weather Knowledge Distillation for Robust 3D Object Detection

Real-Time And Robust 3D Object Detection with Roadside LiDARs

L4DR: LiDAR-4DRadar Fusion for Weather-Robust 3D Object Detection

Semantic Understanding of Foggy Scenes with Purely Synthetic Data

Improved 3D Object Detector Under Snowfall Weather Condition Based on LiDAR Point Cloud

LaNoising: A Data-driven Approach for 903nm ToF LiDAR Performance Modeling under Fog

LiDAR-CS Dataset: LiDAR Point Cloud Dataset with Cross-Sensors for 3D Object Detection

Label-Efficient Semantic Segmentation of LiDAR Point Clouds in Adverse Weather Conditions

Energy-based Detection of Adverse Weather Effects in LiDAR Data