Planning-oriented Autonomous Driving

Yihan Hu,Jiazhi Yang,Li Chen,Keyu Li,Chonghao Sima,Xizhou Zhu,Siqi Chai,Senyao Du,Tianwei Lin,Wenhai Wang,Lewei Lu,Xiaosong Jia,Qiang Liu,Jifeng Dai,Yu Qiao,Hongyang Li

2023-03-24

Abstract:Modern autonomous driving system is characterized as modular tasks in sequential order, i.e., perception, prediction, and planning. In order to perform a wide diversity of tasks and achieve advanced-level intelligence, contemporary approaches either deploy standalone models for individual tasks, or design a multi-task paradigm with separate heads. However, they might suffer from accumulative errors or deficient task coordination. Instead, we argue that a favorable framework should be devised and optimized in pursuit of the ultimate goal, i.e., planning of the self-driving car. Oriented at this, we revisit the key components within perception and prediction, and prioritize the tasks such that all these tasks contribute to planning. We introduce Unified Autonomous Driving (UniAD), a comprehensive framework up-to-date that incorporates full-stack driving tasks in one network. It is exquisitely devised to leverage advantages of each module, and provide complementary feature abstractions for agent interaction from a global perspective. Tasks are communicated with unified query interfaces to facilitate each other toward planning. We instantiate UniAD on the challenging nuScenes benchmark. With extensive ablations, the effectiveness of using such a philosophy is proven by substantially outperforming previous state-of-the-arts in all aspects. Code and models are public.

Computer Vision and Pattern Recognition,Robotics

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to address the issue of modular task design in autonomous driving systems, specifically how to achieve efficient coordination between perception, prediction, and planning. Specifically:

1. **Existing Problems**:
- Current autonomous driving systems typically divide tasks (such as perception, prediction, and planning) into independent modules, which can lead to information loss, error accumulation, and improper feature alignment.
- Although multi-task learning (MTL) can share feature extractors and alleviate these issues to some extent, it may introduce the phenomenon of "negative transfer."

2. **Goals**:
- Propose a unified autonomous driving framework (Unified Autonomous Driving, UniAD) that can integrate complete driving tasks within a single network, including detection, tracking, mapping, motion prediction, and occupancy prediction.
- By optimizing the collaboration between various modules, reduce error accumulation, and ensure that all tasks ultimately serve the planning goal, thereby improving the overall performance and safety of the system.

3. **Core Contributions**:
- Designed a planning-oriented end-to-end framework that connects various modules through a query mechanism, enabling flexible task interaction and knowledge transfer.
- Validated the significant advantages of UniAD in all key metrics on the nuScenes benchmark, demonstrating the effectiveness of this approach.

In summary, the main purpose of this paper is to solve the issue of modular task design in existing systems by designing a more efficient and unified autonomous driving framework to enhance the overall performance and safety of autonomous driving.