Abstract:Existing LiDAR semantic segmentation methods often struggle with performance declines in adverse weather conditions. Previous work has addressed this issue by simulating adverse weather or employing universal data augmentation during training. However, these methods lack a detailed analysis and understanding of how adverse weather negatively affects LiDAR semantic segmentation performance. Motivated by this issue, we identified key factors of adverse weather and conducted a toy experiment to pinpoint the main causes of performance degradation: (1) Geometric perturbation due to refraction caused by fog or droplets in the air and (2) Point drop due to energy absorption and occlusions. Based on these findings, we propose new strategic data augmentation techniques. First, we introduced a Selective Jittering (SJ) that jitters points in the random range of depth (or angle) to mimic geometric perturbation. Additionally, we developed a Learnable Point Drop (LPD) to learn vulnerable erase patterns with a Deep Q-Learning Network to approximate the point drop phenomenon from adverse weather conditions. Without precise weather simulation, these techniques strengthen the LiDAR semantic segmentation model by exposing it to vulnerable conditions identified by our data-centric analysis. Experimental results confirmed the suitability of the proposed data augmentation methods for enhancing robustness against adverse weather conditions. Our method achieves a notable 39.5 mIoU on the SemanticKITTI-to-SemanticSTF benchmark, improving the baseline by 8.1\%p and establishing a new state-of-the-art. Our code will be released at \url{<a class="link-external link-https" href="https://github.com/engineerJPark/LiDARWeather" rel="external noopener nofollow">this https URL</a>}.

Adverse Weather Conditions Augmentation of LiDAR Scenes with Latent Diffusion Models

LiDAR Point Cloud Augmentation for Adverse Conditions Using Conditional Generative Model

Energy-based Detection of Adverse Weather Effects in LiDAR Data

Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection

Rethinking Data Augmentation for Robust LiDAR Semantic Segmentation in Adverse Weather

GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation

SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Models

Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion

Latent diffusion models for parameterization of facies-based geomodels and their use in data assimilation

UniMix: Towards Domain Adaptive and Generalizable LiDAR Semantic Segmentation in Adverse Weather

Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification

Latent diffusion models for parameterization and data assimilation of facies-based geomodels

Towards Realistic Scene Generation with LiDAR Diffusion Models

Adver-City: Open-Source Multi-Modal Dataset for Collaborative Perception Under Adverse Weather Conditions

Realistic Rainy Weather Simulation for LiDARs in CARLA Simulator

Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time

LidarDM: Generative LiDAR Simulation in a Generated World

Data Augmentation of Automotive LIDAR Point Clouds under Adverse Weather Situations

Label-Efficient Semantic Segmentation of LiDAR Point Clouds in Adverse Weather Conditions

L-DIG: A GAN-Based Method for LiDAR Point Cloud Processing under Snow Driving Conditions

RangeLDM: Fast Realistic LiDAR Point Cloud Generation