ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Driving

Han Lu,Xiaosong Jia,Yichen Xie,Wenlong Liao,Xiaokang Yang,Junchi Yan

2024-03-05

Abstract:End-to-end differentiable learning for autonomous driving (AD) has recently become a prominent paradigm. One main bottleneck lies in its voracious appetite for high-quality labeled data e.g. 3D bounding boxes and semantic segmentation, which are notoriously expensive to manually annotate. The difficulty is further pronounced due to the prominent fact that the behaviors within samples in AD often suffer from long tailed distribution. In other words, a large part of collected data can be trivial (e.g. simply driving forward in a straight road) and only a few cases are safety-critical. In this paper, we explore a practically important yet under-explored problem about how to achieve sample and label efficiency for end-to-end AD. Specifically, we design a planning-oriented active learning method which progressively annotates part of collected raw data according to the proposed diversity and usefulness criteria for planning routes. Empirically, we show that our planning-oriented approach could outperform general active learning methods by a large margin. Notably, our method achieves comparable performance with state-of-the-art end-to-end AD methods - by using only 30% nuScenes data. We hope our work could inspire future works to explore end-to-end AD from a data-centric perspective in addition to methodology efforts.

Computer Vision and Pattern Recognition,Artificial Intelligence,Robotics

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

The paper aims to address the issue of low data annotation efficiency in End-to-End Autonomous Driving (E2E-AD). Specifically, the paper focuses on the following aspects:

1. **High Data Annotation Cost**: High-quality annotated data (such as 3D bounding boxes and semantic segmentation) is very expensive and difficult to obtain in autonomous driving.
2. **Long-Tail Distribution of Data**: A large amount of collected data is relatively simple (e.g., driving on straight roads), while critical safety-related cases are scarce, limiting the effectiveness of data-driven methods.
3. **Limitations of Existing Active Learning Methods**: Existing active learning methods are usually only applicable to single-modal data (such as images) and are mainly targeted at classification tasks. However, autonomous driving tasks are more complex, involving multi-modal information (such as video streams, driving trajectories, etc.).

To address the above issues, the paper proposes a Planning-Oriented Active Learning method, called ActiveAD. This method selects the most useful data samples for annotation by designing specific diversity and uncertainty metrics, thereby optimizing the planning task. Experimental results show that ActiveAD can achieve performance comparable to or even better than the current best methods with a small amount of data (e.g., 30% of annotated data).