Abstract:With the continuous maturation and application of autonomous driving technology, a systematic examination of open-source autonomous driving datasets becomes instrumental in fostering the robust evolution of the industry ecosystem. Current autonomous driving datasets can broadly be categorized into two generations. The first-generation autonomous driving datasets are characterized by relatively simpler sensor modalities, smaller data scale, and is limited to perception-level tasks. KITTI, introduced in 2012, serves as a prominent representative of this initial wave. In contrast, the second-generation datasets exhibit heightened complexity in sensor modalities, greater data scale and diversity, and an expansion of tasks from perception to encompass prediction and control. Leading examples of the second generation include nuScenes and Waymo, introduced around 2019. This comprehensive review, conducted in collaboration with esteemed colleagues from both academia and industry, systematically assesses over seventy open-source autonomous driving datasets from domestic and international sources. It offers insights into various aspects, such as the principles underlying the creation of high-quality datasets, the pivotal role of data engine systems, and the utilization of generative foundation models to facilitate scalable data generation. Furthermore, this review undertakes an exhaustive analysis and discourse regarding the characteristics and data scales that future third-generation autonomous driving datasets should possess. It also delves into the scientific and technical challenges that warrant resolution. These endeavors are pivotal in advancing autonomous innovation and fostering technological enhancement in critical domains. For further details, please refer to <a class="link-external link-https" href="https://github.com/OpenDriveLab/DriveAGI" rel="external noopener nofollow">this https URL</a>.

Processing, assessing, and enhancing the Waymo autonomous vehicle open dataset for driving behavior research

A Review on Trajectory Datasets on Advanced Driver Assistance System

Autonomous Vehicle’s Impact on Traffic: Empirical Evidence From Waymo Open Dataset and Implications From Modelling

A unified longitudinal trajectory dataset for automated vehicle

ULTra-AV: A Unified Longitudinal Trajectory Dataset for Automated Vehicle

Large Car-following Data Based on Lyft level-5 Open Dataset: Following Autonomous Vehicles vs. Human-driven Vehicles

Driving Behaviour Style Study with a Hybrid Deep Learning Framework Based on GPS Data

Automated Vehicle-Involved Traffic Flow Studies: A Survey of Assumptions, Models, Speculations, and Perspectives

Scalability in Perception for Autonomous Driving: Waymo Open Dataset.

Characterizing Behavioral Differences of Autonomous Vehicles and Human-Driven Vehicles at Signalized Intersections Based on Waymo Open Dataset

A Survey on Datasets for Decision-making of Autonomous Vehicle

A Survey on Datasets for the Decision Making of Autonomous Vehicles

Human Observation-Inspired Trajectory Prediction for Autonomous Driving in Mixed-Autonomy Traffic Environments

Data-driven Traffic Simulation: A Comprehensive Review

Vehicle-Human Interactive Behaviors in Emergency: Data Extraction from Traffic Accident Videos

AVOID: Autonomous Vehicle Operation Incident Dataset Across the Globe

Energetic Impacts of Autonomous Vehicles in Real-World Traffic Conditions From Nine Open-Source Datasets

Exploring Driving Behavior for Autonomous Vehicles Based on Gramian Angular Field Vision Transformer

Modeling Autonomous Vehicles’ Altruistic Behavior to Human-Driven Vehicles in the Car following Events and Impact Analysis

Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future

DAnoScenE: a driving anomaly scenario extraction framework for autonomous vehicles in urban streets