Abstract:Recent advances in deep learning have markedly improved autonomous driving (AD) models, particularly end-to-end systems that integrate perception, prediction, and planning stages, achieving state-of-the-art performance. However, these models remain vulnerable to adversarial attacks, where human-imperceptible perturbations can disrupt decision-making processes. While adversarial training is an effective method for enhancing model robustness against such attacks, no prior studies have focused on its application to end-to-end AD models. In this paper, we take the first step in adversarial training for end-to-end AD models and present a novel Module-wise Adaptive Adversarial Training (MA2T). However, extending conventional adversarial training to this context is highly non-trivial, as different stages within the model have distinct objectives and are strongly interconnected. To address these challenges, MA2T first introduces Module-wise Noise Injection, which injects noise before the input of different modules, targeting training models with the guidance of overall objectives rather than each independent module loss. Additionally, we introduce Dynamic Weight Accumulation Adaptation, which incorporates accumulated weight changes to adaptively learn and adjust the loss weights of each module based on their contributions (accumulated reduction rates) for better balance and robust training. To demonstrate the efficacy of our defense, we conduct extensive experiments on the widely-used nuScenes dataset across several end-to-end AD models under both white-box and black-box attacks, where our method outperforms other baselines by large margins (+5-10%). Moreover, we validate the robustness of our defense through closed-loop evaluation in the CARLA simulation environment, showing improved resilience even against natural corruption.

Adversarial Attack Against Urban Scene Segmentation for Autonomous Vehicles

Adversarial Robustness Analysis of LiDAR-included Models in Autonomous Driving

SR-Adv: Salient Region Adversarial Attacks on 3D Point Clouds for Autonomous Driving

Attacking vision-based perception in end-to-end autonomous driving models

An Adversarial Attack on Salient Regions of Traffic Sign

Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks

Evaluating Adversarial Attacks on Driving Safety in Vision-Based Autonomous Vehicles

Efficient Adversarial Attack Strategy Against 3D Object Detection in Autonomous Driving Systems

Learning When to Use Adaptive Adversarial Image Perturbations against Autonomous Vehicles

Attack End-to-End Autonomous Driving through Module-Wise Noise

Adversarial Point Cloud Perturbations Against 3D Object Detection in Autonomous Driving Systems

Enhancing Robustness of Indoor Robotic Navigation with Free-Space Segmentation Models Against Adversarial Attacks

Adversarial Driving: Attacking End-to-End Autonomous Driving

Module-wise Adaptive Adversarial Training for End-to-end Autonomous Driving

Exploring Adversarial Robustness of LiDAR Semantic Segmentation in Autonomous Driving

Assessing the Threat of Adversarial Examples on Deep Neural Networks for Remote Sensing Scene Classification: Attacks and Defenses

Robust Roadside Physical Adversarial Attack Against Deep Learning in Lidar Perception Modules

Evaluating the Robustness of Off-Road Autonomous Driving Segmentation against Adversarial Attacks: A Dataset-Centric analysis

Towards Improving the Anti-attack Capability of the RangeNet++.

Towards Improving the Anti-attack Capability of the RangeNet++

Adversarial Attacks on Video Object Segmentation with Hard Region Discovery