Abstract:Autonomous driving technology has advanced significantly with deep learning, but noise and attacks threaten its real-world deployment. While research has revealed vulnerabilities in individual intelligent tasks, a comprehensive evaluation of these impacts across complete end-to-end systems is still underexplored. To address this void, we thoroughly analyze the robustness of four end-to-end autonomous driving systems against various noise and build the RobustE2E Benchmark, including five traditional adversarial attacks and a newly proposed Module-Wise Attack specifically targeting end-to-end autonomous driving in white-box settings, as well as four major categories of natural corruptions (a total of 17 types, with five severity levels) in black-box settings. Additionally, we extend the robustness evaluation from the open-loop model level to the closed-loop case studies of autonomous driving system level. Our comprehensive evaluation and analysis provide valuable insights into the robustness of end-to-end autonomous driving, which may offer potential guidance for targeted improvements to models. For example, (1) even the most advanced end-to-end models suffer large planning failures under minor perturbations, with perception tasks showing the most substantial decline; (2) among adversarial attacks, our Module-Wise Attack poses the greatest threat to end-to-end autonomous driving models, while PGD-l2 is the weakest, and among four categories of natural corruptions, noise and weather are the most harmful, followed by blur and digital distortion being less severe; (3) the integrated, multitask approach results in significantly higher robustness and reliability compared with the simpler design, highlighting the critical role of collaborative multitask in autonomous driving; and (4) the autonomous driving systems amplify the model's lack of robustness, etc. Our research contributes to developing more resilient autonomous driving models and their deployment in the real world.

BARS: Local Robustness Certification for Deep Learning Based Traffic Analysis Systems.

Uniformity of Markov Elements in Deep Reinforcement Learning for Traffic Signal Control

Optimal Smoothing Distribution Exploration for Backdoor Neutralization in Deep Learning-based Traffic Systems

Certified Adversarial Robustness Under the Bounded Support Set.

Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review

On Certifying Robustness against Backdoor Attacks via Randomized Smoothing

Robustness Certification Research on Deep Learning Models:A Survey

Improving model robustness of traffic crash risk evaluation via adversarial mix-up under traffic flow fundamental diagram

SoK: Certified Robustness for Deep Neural Networks

Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond

RBLJAN: Robust Byte-Label Joint Attention Network for Network Traffic Classification

Analyzing Robustness of the Deep Reinforcement Learning Algorithm in Ramp Metering Applications Considering False Data Injection Attack and Defense

Robust Spatiotemporal Traffic Forecasting with Reinforced Dynamic Adversarial Training

Evaluating and Improving Adversarial Robustness of Machine Learning-Based Network Intrusion Detectors

A Visual Analytics System for Improving Attention-based Traffic Forecasting Models

SITAR: Evaluating the Adversarial Robustness of Traffic Light Recognition in Level-4 Autonomous Driving

Adversarial Network Traffic: Towards Evaluating the Robustness of Deep Learning-Based Network Traffic Classification

RAB: Provable Robustness Against Backdoor Attacks

DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothing

RobOT: Robustness-Oriented Testing for Deep Learning Systems

RobustE2E: Exploring the Robustness of End-to-End Autonomous Driving