Abstract:Recently, LiDAR perception methods for autonomous vehicles, powered by deep neural networks have experienced steep growth in performance on classic benchmarks, such as nuScenes and SemanticKITTI. However, there are still large gaps in performance when deploying models trained on such single-sensor setups to modern multi-sensor vehicles. In this work, we investigate if a lack of invariance may be responsible for these performance gaps, and propose some initial solutions in the form of application-specific data augmentations, which can facilitate better transfer to multi-sensor LiDAR setups. We provide experimental evidence that our proposed augmentations improve generalization across LiDAR sensor setups, and investigate how these augmentations affect the models' invariance properties on simulations of different LiDAR sensor setups.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of performance degradation of LiDAR (Light Detection and Ranging) perception models in autonomous vehicles when they are deployed in modern multi - sensor vehicles after being trained on single - sensor datasets. Specifically, the authors explore the performance gap caused by the lack of invariance and propose to improve this invariance by applying specific data augmentation methods, thereby enhancing the generalization ability of the model for multi - sensor LiDAR settings. #### Main problems 1. **Performance gap**: Existing LiDAR perception models perform well on classic benchmarks such as nuScenes and SemanticKITTI, but their performance drops significantly when deployed in modern vehicles equipped with multiple LiDAR sensors. 2. **Lack of invariance**: These models perform poorly when processing multi - sensor fused point clouds, probably because they lack sufficient invariance to data transformations under different sensor settings. 3. **High annotation cost**: The cost of re - annotating multi - sensor data for retraining is high, so a method that can improve the generalization ability of the model without re - annotating is required. #### Solutions The authors propose and validate several new data augmentation methods to improve the invariance and generalization ability of the model for multi - sensor LiDAR settings. Specifically: - **Frustum Drop Augmentation**: By randomly dropping points within the frustum to simulate changes in the field of view, the robustness of the model to changes in the field of view is improved. - **Mis - Calibration Augmentation**: By copying, randomly translating and rotating point clouds to simulate multi - sensor overlapping scan patterns, the robustness of the model to changes in point density is improved. In addition, the authors introduce a new metric - Normalized Feature Similarity (NFS) - to quantify invariance at the feature level. ### Summary The main goal of this paper is to bridge the generalization gap between single - sensor datasets and modern multi - sensor settings by improving the invariance of the model, thereby enhancing the performance of LiDAR perception models in autonomous vehicles.

From One to the Power of Many: Augmentations for Invariance to Multi-LiDAR Perception from Single-Sensor Datasets

Structuring a Training Strategy to Robustify Perception Models with Realistic Image Augmentations

How Important are Data Augmentations to Close the Domain Gap for Object Detection in Orbit?

Quantifying Data Augmentation for LiDAR based 3D Object Detection

NeRFmentation: NeRF-based Augmentation for Monocular Depth Estimation

What Can be Seen is What You Get: Structure Aware Point Cloud Augmentation

UniLiDAR: Bridge the Domain Gap among Different LiDARs for Continual Learning

D-Aug: Enhancing Data Augmentation for Dynamic LiDAR Scenes

3D Data Augmentation for Driving Scenes on Camera

Performance Enhancement Using Data Augmentation of Point Cloud Based 3D Object Detection for Autonomous Driving

Just Add $100 More: Augmenting NeRF-based Pseudo-LiDAR Point Cloud for Resolving Class-imbalance Problem

Instant Domain Augmentation for LiDAR Semantic Segmentation

Influence of Camera-LiDAR Configuration on 3D Object Detection for Autonomous Driving

Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation

Improving Deep Learning for Maritime Remote Sensing through Data Augmentation and Latent Space

Scalable Primitives for Generalized Sensor Fusion in Autonomous Vehicles

Sem-Aug: Improving Camera-LiDAR Feature Fusion With Semantic Augmentation for 3D Vehicle Detection

Data Augmentation of Automotive LIDAR Point Clouds under Adverse Weather Situations

Uni-to-Multi Modal Knowledge Distillation for Bidirectional LiDAR-Camera Semantic Segmentation

LiDAR Point Cloud Augmentation for Adverse Conditions Using Conditional Generative Model

From One to Many: Dynamic Cross Attention Networks for LiDAR and Camera Fusion