Abstract:Existing LiDAR semantic segmentation methods often struggle with performance declines in adverse weather conditions. Previous work has addressed this issue by simulating adverse weather or employing universal data augmentation during training. However, these methods lack a detailed analysis and understanding of how adverse weather negatively affects LiDAR semantic segmentation performance. Motivated by this issue, we identified key factors of adverse weather and conducted a toy experiment to pinpoint the main causes of performance degradation: (1) Geometric perturbation due to refraction caused by fog or droplets in the air and (2) Point drop due to energy absorption and occlusions. Based on these findings, we propose new strategic data augmentation techniques. First, we introduced a Selective Jittering (SJ) that jitters points in the random range of depth (or angle) to mimic geometric perturbation. Additionally, we developed a Learnable Point Drop (LPD) to learn vulnerable erase patterns with a Deep Q-Learning Network to approximate the point drop phenomenon from adverse weather conditions. Without precise weather simulation, these techniques strengthen the LiDAR semantic segmentation model by exposing it to vulnerable conditions identified by our data-centric analysis. Experimental results confirmed the suitability of the proposed data augmentation methods for enhancing robustness against adverse weather conditions. Our method achieves a notable 39.5 mIoU on the SemanticKITTI-to-SemanticSTF benchmark, improving the baseline by 8.1\%p and establishing a new state-of-the-art. Our code will be released at \url{<a class="link-external link-https" href="https://github.com/engineerJPark/LiDARWeather" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problem of the performance degradation of LiDAR (Light Detection and Ranging) semantic segmentation models under adverse weather conditions. Specifically: 1. **Existing challenges**: - **Performance degradation**: Existing LiDAR semantic segmentation methods perform poorly and their performance drops significantly under adverse weather conditions (such as snow, fog, rain or wet surfaces). - **Lack of detailed analysis**: Previous works addressed this problem by simulating adverse weather or using general data augmentation methods during the training process, but these methods lack detailed analysis and understanding of how adverse weather negatively affects the performance of LiDAR semantic segmentation. 2. **Research motivation**: - The authors identified the main influencing factors of adverse weather through a data - driven approach and conducted experiments to determine the main causes of performance degradation: 1. **Geometric perturbation**: Refraction caused by fog or water droplets in the air. 2. **Point loss**: Due to energy absorption and occlusion. 3. **Solutions**: - Based on the above findings, the authors proposed two new data augmentation techniques: 1. **Selective Jittering (SJ)**: Simulate geometric perturbation by jittering the depth or angle within a random range. 2. **Learnable Point Drop (LPD)**: Use a Deep Q - Learning Network (DQN) to learn vulnerable erasure patterns to approximate the point loss phenomenon under adverse weather conditions. 4. **Objective**: - By introducing these data augmentation techniques, improve the robustness of LiDAR semantic segmentation models under adverse weather conditions without relying on accurate weather simulation. 5. **Results**: - The experimental results show that the proposed data augmentation methods significantly improve the robustness of the model. In particular, in the SemanticKITTI to SemanticSTF benchmark test, the mIoU is increased by 8.1% and a new state - of - the - art level is established. In summary, this paper solves the problem of performance degradation of existing LiDAR semantic segmentation models under adverse weather conditions by in - depth analysis of the impact of adverse weather on LiDAR data and proposing targeted data augmentation methods.

Rethinking Data Augmentation for Robust LiDAR Semantic Segmentation in Adverse Weather

Label-Efficient Semantic Segmentation of LiDAR Point Clouds in Adverse Weather Conditions

Weather-aware autopilot: Domain generalization for point cloud semantic segmentation in diverse weather scenarios

Semantic Segmentation under Adverse Conditions: A Weather and Nighttime-aware Synthetic Data-based Approach

Energy-based Detection of Adverse Weather Effects in LiDAR Data

WeatherProof: Leveraging Language Guidance for Semantic Segmentation in Adverse Weather

Enhancing Autonomous Vehicle Perception in Adverse Weather through Image Augmentation during Semantic Segmentation Training

Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection

LiDAR Point Cloud Augmentation for Adverse Conditions Using Conditional Generative Model

Mastering adverse weather: a two-stage approach for robust semantic segmentation in autonomous driving

3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds

WeatherProof: A Paired-Dataset Approach to Semantic Segmentation in Adverse Weather

Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time

Instant Domain Augmentation for LiDAR Semantic Segmentation

Data Augmentation of Automotive LIDAR Point Clouds under Adverse Weather Situations

Leveraging Smooth Deformation Augmentation for LiDAR Point Cloud Semantic Segmentation

An improved point cloud denoising method in adverse weather conditions based on PP-LiteSeg network

Realistic Rainy Weather Simulation for LiDARs in CARLA Simulator

Improved 3D Object Detector Under Snowfall Weather Condition Based on LiDAR Point Cloud

Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

Learning from Spatio-temporal Correlation for Semi-Supervised LiDAR Semantic Segmentation