Evaluating the Robustness of Off-Road Autonomous Driving Segmentation against Adversarial Attacks: A Dataset-Centric analysis

Pankaj Deoli,Rohit Kumar,Axel Vierling,Karsten Berns

2024-02-03

Abstract:This study investigates the vulnerability of semantic segmentation models to adversarial input perturbations, in the domain of off-road autonomous driving. Despite good performance in generic conditions, the state-of-the-art classifiers are often susceptible to (even) small perturbations, ultimately resulting in inaccurate predictions with high confidence. Prior research has directed their focus on making models more robust by modifying the architecture and training with noisy input images, but has not explored the influence of datasets in adversarial attacks. Our study aims to address this gap by examining the impact of non-robust features in off-road datasets and comparing the effects of adversarial attacks on different segmentation network architectures. To enable this, a robust dataset is created consisting of only robust features and training the networks on this robustified dataset. We present both qualitative and quantitative analysis of our findings, which have important implications on improving the robustness of machine learning models in off-road autonomous driving applications. Additionally, this work contributes to the safe navigation of autonomous robot Unimog U5023 in rough off-road unstructured environments by evaluating the robustness of segmentation outputs. The code is publicly available at

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper primarily explores the vulnerability of semantic segmentation models to adversarial input perturbations in the off-road autonomous driving domain. Although current state-of-the-art classifiers perform well under general conditions, they are often susceptible to (even minor) perturbations, leading to inaccurate predictions with high confidence.

Specifically, the paper focuses on the following aspects:
1. **Impact of Adversarial Attacks**: Investigating the impact of adversarial attacks on semantic segmentation models in off-road autonomous driving scenarios and exploring how improving the dataset can enhance the robustness of the models.
2. **Role of Datasets**: Existing research mainly focuses on improving model robustness by modifying model architectures or training with noisy images, but has not sufficiently explored the role of datasets in adversarial attacks. This paper aims to fill this gap by analyzing the impact of non-robust features in off-road datasets and comparing the performance of different segmentation network architectures under adversarial attacks.
3. **Creation of Robust Datasets**: Creating a dataset that contains only robust features and training networks on this dataset to evaluate its effectiveness in improving model robustness.

Through in-depth research on these aspects, the paper aims to enhance the robustness of machine learning models in off-road autonomous driving applications and ultimately achieve safe navigation of the autonomous robot Unimog U5023 in complex off-road environments.