Abstract:We propose UAD, a method for vision-based end-to-end autonomous driving (E2EAD), achieving the best open-loop evaluation performance in nuScenes, meanwhile showing robust closed-loop driving quality in CARLA. Our motivation stems from the observation that current E2EAD models still mimic the modular architecture in typical driving stacks, with carefully designed supervised perception and prediction subtasks to provide environment information for oriented planning. Although achieving groundbreaking progress, such design has certain drawbacks: 1) preceding subtasks require massive high-quality 3D annotations as supervision, posing a significant impediment to scaling the training data; 2) each submodule entails substantial computation overhead in both training and inference. To this end, we propose UAD, an E2EAD framework with an unsupervised proxy to address all these issues. Firstly, we design a novel Angular Perception Pretext to eliminate the annotation requirement. The pretext models the driving scene by predicting the angular-wise spatial objectness and temporal dynamics, without manual annotation. Secondly, a self-supervised training strategy, which learns the consistency of the predicted trajectories under different augment views, is proposed to enhance the planning robustness in steering scenarios. Our UAD achieves 38.7% relative improvements over UniAD on the average collision rate in nuScenes and surpasses VAD for 41.32 points on the driving score in CARLA's Town05 Long benchmark. Moreover, the proposed method only consumes 44.3% training resources of UniAD and runs 3.4 times faster in inference. Our innovative design not only for the first time demonstrates unarguable performance advantages over supervised counterparts, but also enjoys unprecedented efficiency in data, training, and inference. Code and models will be released at <a class="link-external link-https" href="https://github.com/KargoBot_Research/UAD" rel="external noopener nofollow">this https URL</a>.

Nuscenescomplex: A More Rigorous Evaluation Framework for End-to-End Autonomous Driving Planning

Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

Towards learning-based planning:The nuPlan benchmark for real-world autonomous driving

Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?

nuScenes: A Multimodal Dataset for Autonomous Driving

Planning-oriented Autonomous Driving

Planning by Simulation: Motion Planning with Learning-based Parallel Scenario Prediction for Autonomous Driving

ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling

On Offline Evaluation of 3D Object Detection for Autonomous Driving

SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation

The efficacy of Neural Planning Metrics: A meta-analysis of PKL on nuScenes

S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generation

End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotation

Multi-Modal Sensor Fusion-Based Deep Neural Network for End-to-End Autonomous Driving With Scene Understanding

Enhanced Scene Understanding and Situation Awareness for Autonomous Vehicles Based on Semantic Segmentation

Complexity Quantification of Driving Scenarios with Dynamic Evolution Characteristics

SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries