Abstract:Recent advances in deep learning have markedly improved autonomous driving (AD) models, particularly end-to-end systems that integrate perception, prediction, and planning stages, achieving state-of-the-art performance. However, these models remain vulnerable to adversarial attacks, where human-imperceptible perturbations can disrupt decision-making processes. While adversarial training is an effective method for enhancing model robustness against such attacks, no prior studies have focused on its application to end-to-end AD models. In this paper, we take the first step in adversarial training for end-to-end AD models and present a novel Module-wise Adaptive Adversarial Training (MA2T). However, extending conventional adversarial training to this context is highly non-trivial, as different stages within the model have distinct objectives and are strongly interconnected. To address these challenges, MA2T first introduces Module-wise Noise Injection, which injects noise before the input of different modules, targeting training models with the guidance of overall objectives rather than each independent module loss. Additionally, we introduce Dynamic Weight Accumulation Adaptation, which incorporates accumulated weight changes to adaptively learn and adjust the loss weights of each module based on their contributions (accumulated reduction rates) for better balance and robust training. To demonstrate the efficacy of our defense, we conduct extensive experiments on the widely-used nuScenes dataset across several end-to-end AD models under both white-box and black-box attacks, where our method outperforms other baselines by large margins (+5-10%). Moreover, we validate the robustness of our defense through closed-loop evaluation in the CARLA simulation environment, showing improved resilience even against natural corruption.

Multi-view Defense with Adversarial Autoencoders.

Adversarial Attacks for Multi-view Deep Models

Adversarial Robustness and Attacks for Multi-View Deep Models.

MV2MAE: Multi-View Video Masked Autoencoders

Towards Model-Agnostic Adversarial Defenses using Adversarially Trained Autoencoders

Module-wise Adaptive Adversarial Training for End-to-end Autonomous Driving

Defending against adversarial attacks using spherical sampling-based variational auto-encoder

Double Backpropagation for Training Autoencoders against Adversarial Attack

Multi-view Representation Learning from Malware to Defend Against Adversarial Variants

Teaching Masked Autoencoder With Strong Augmentations

MAD-VAE: Manifold Awareness Defense Variational Autoencoder

Masked Two-channel Decoupling Framework for Incomplete Multi-view Weak Multi-label Learning

Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks

Multi-View Transformation Via Mutual-Encoding InfoGenerative Adversarial Networks

Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks

Multi-view Adversarial Discriminator: Mine the Non-causal Factors for Object Detection in Unseen Domains

A View-Adversarial Framework for Multi-View Network Embedding

MULDEF: Multi-model-based Defense Against Adversarial Examples for Neural Networks

AutoMA: Towards Automatic Model Augmentation for Transferable Adversarial Attacks

A tutorial on multi-view autoencoders using the multi-view-AE library