Abstract:Autonomous driving technology has advanced significantly with deep learning, but noise and attacks threaten its real-world deployment. While research has revealed vulnerabilities in individual intelligent tasks, a comprehensive evaluation of these impacts across complete end-to-end systems is still underexplored. To address this void, we thoroughly analyze the robustness of four end-to-end autonomous driving systems against various noise and build the RobustE2E Benchmark, including five traditional adversarial attacks and a newly proposed Module-Wise Attack specifically targeting end-to-end autonomous driving in white-box settings, as well as four major categories of natural corruptions (a total of 17 types, with five severity levels) in black-box settings. Additionally, we extend the robustness evaluation from the open-loop model level to the closed-loop case studies of autonomous driving system level. Our comprehensive evaluation and analysis provide valuable insights into the robustness of end-to-end autonomous driving, which may offer potential guidance for targeted improvements to models. For example, (1) even the most advanced end-to-end models suffer large planning failures under minor perturbations, with perception tasks showing the most substantial decline; (2) among adversarial attacks, our Module-Wise Attack poses the greatest threat to end-to-end autonomous driving models, while PGD-l2 is the weakest, and among four categories of natural corruptions, noise and weather are the most harmful, followed by blur and digital distortion being less severe; (3) the integrated, multitask approach results in significantly higher robustness and reliability compared with the simpler design, highlighting the critical role of collaborative multitask in autonomous driving; and (4) the autonomous driving systems amplify the model's lack of robustness, etc. Our research contributes to developing more resilient autonomous driving models and their deployment in the real world.

Evaluating the Robustness of Object Detection in Autonomous Driving System

Robustness-Aware 3D Object Detection in Autonomous Driving: A Review and Outlook

Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving

Robust Object Detection for Autonomous Driving Based on Semi-Supervised Learning

Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

RobustE2E: Exploring the Robustness of End-to-End Autonomous Driving

Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving

Uncertainty Evaluation of Object Detection Algorithms for Autonomous Vehicles

Automated Evaluation of Semantic Segmentation Robustness for Autonomous Driving

Evaluating Robustness over High Level Driving Instruction for Autonomous Driving

Testing Autonomous Driving System Based on Scenic

3D Object Detection for Autonomous Driving: A Survey

On the Robustness of Object Detection Models in Aerial Images

Robustness Enhancement of Object Detection in Advanced Driver Assistance Systems (ADAS)

Confidence-Aware Object Detection Based on MobileNetv2 for Autonomous Driving

3D Object Detection for Autonomous Driving: A Comprehensive Survey

Common Corruptions for Enhancing and Evaluating Robustness in Air-to-Air Visual Object Detection

Quality Assessment of Image Dataset for Autonomous Driving.

Robust LiDAR-Based Vehicle Detection for On-Road Autonomous Driving

Towards cross-task universal perturbation against black-box object detectors in autonomous driving