Abstract:With the continuous maturation and application of autonomous driving technology, a systematic examination of open-source autonomous driving datasets becomes instrumental in fostering the robust evolution of the industry ecosystem. Current autonomous driving datasets can broadly be categorized into two generations. The first-generation autonomous driving datasets are characterized by relatively simpler sensor modalities, smaller data scale, and is limited to perception-level tasks. KITTI, introduced in 2012, serves as a prominent representative of this initial wave. In contrast, the second-generation datasets exhibit heightened complexity in sensor modalities, greater data scale and diversity, and an expansion of tasks from perception to encompass prediction and control. Leading examples of the second generation include nuScenes and Waymo, introduced around 2019. This comprehensive review, conducted in collaboration with esteemed colleagues from both academia and industry, systematically assesses over seventy open-source autonomous driving datasets from domestic and international sources. It offers insights into various aspects, such as the principles underlying the creation of high-quality datasets, the pivotal role of data engine systems, and the utilization of generative foundation models to facilitate scalable data generation. Furthermore, this review undertakes an exhaustive analysis and discourse regarding the characteristics and data scales that future third-generation autonomous driving datasets should possess. It also delves into the scientific and technical challenges that warrant resolution. These endeavors are pivotal in advancing autonomous innovation and fostering technological enhancement in critical domains. For further details, please refer to <a class="link-external link-https" href="https://github.com/OpenDriveLab/DriveAGI" rel="external noopener nofollow">this https URL</a>.

Towards Scenario- and Capability-Driven Dataset Development and Evaluation: An Approach in the Context of Mapless Automated Driving

Is it Safe to Drive? An Overview of Factors, Metrics, and Datasets for Driveability Assessment in Autonomous Driving

Is it Safe to Drive? An Overview of Factors, Challenges, and Datasets for Driveability Assessment in Autonomous Driving

Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future

DeepScenario: An Open Driving Scenario Dataset for Autonomous Driving System Testing

Towards Sensor Data Abstraction of Autonomous Vehicle Perception Systems

A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook

D2E-An Autonomous Decision-making Dataset involving Driver States and Human Evaluation

A Survey on Datasets for the Decision Making of Autonomous Vehicles

ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling

A Survey on Datasets for Decision-making of Autonomous Vehicle

Scalability in Perception for Autonomous Driving: Waymo Open Dataset.

The ADUULM-360 Dataset -- A Multi-Modal Dataset for Depth Estimation in Adverse Weather

Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

Automated Vehicle Driver Monitoring Dataset from Real-World Scenarios

Proximity based automatic data annotation for autonomous driving

Towards Scenario Retrieval of Real Driving Data with Large Vision-Language Models

The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems

The ApolloScape Open Dataset for Autonomous Driving and its Application

Synthetic Datasets for Autonomous Driving: A Survey

SCOPE: A Synthetic Multi-Modal Dataset for Collective Perception Including Physical-Correct Weather Conditions