AADS: Augmented autonomous driving simulation using data-driven algorithms

W. Li,C. W. Pan,R. Zhang,J. P. Ren,J. Fang,F. L. Yan,Q. C. Geng,X. Y. Huang,H. J. Gong,W. W. Xu,G. P. Wang,D. Manocha,R. G. Yang,Y. X. Ma

DOI: https://doi.org/10.1126/scirobotics.aaw0863

IF: 25

2019-03-13

Science Robotics

Abstract:A scalable simulation system combines augmented images with synthesized traffic to enable training and testing of autonomous driving.

robotics

What problem does this paper attempt to address?

AADS: Augmented autonomous driving simulation using data-driven algorithms

W. Li,C. W. Pan,R. Zhang,J. P. Ren,J. Fang,F. L. Yan,Q. C. Geng,X. Y. Huang,H. J. Gong,W. W. Xu,G. P. Wang,D. Manocha,R. G. Yang,Y. X. Ma

DOI: https://doi.org/10.1126/scirobotics.aaw0863

IF: 25

2019-03-13

Science Robotics

Abstract:A scalable simulation system combines augmented images with synthesized traffic to enable training and testing of autonomous driving.

robotics
Augmented LiDAR Simulator for Autonomous Driving

Jin Fang,Dingfu Zhou,Feilong Yan,Tongtong Zhao,Feihu Zhang,Yu Ma,Liang Wang,Ruigang Yang

DOI: https://doi.org/10.1109/LRA.2020.2969927

2019-04-10

Abstract:In Autonomous Driving (AD), detection and tracking of obstacles on the roads is a critical task. Deep-learning based methods using annotated LiDAR data have been the most widely adopted approach for this. Unfortunately, annotating 3D point cloud is a very challenging, time- and money-consuming task. In this paper, we propose a novel LiDAR simulator that augments real point cloud with synthetic obstacles (e.g., cars, pedestrians, and other movable objects). Unlike previous simulators that entirely rely on CG models and game engines, our augmented simulator bypasses the requirement to create high-fidelity background CAD models. Instead, we can simply deploy a vehicle with a LiDAR scanner to sweep the street of interests to obtain the background point cloud, based on which annotated point cloud can be automatically generated. This unique "scan-and-simulate" capability makes our approach scalable and practical, ready for large-scale industrial applications. In this paper, we describe our simulator in detail, in particular the placement of obstacles that is critical for performance enhancement. We show that detectors with our simulated LiDAR point cloud alone can perform comparably (within two percentage points) with these trained with real data. Mixing real and simulated data can achieve over 95% accuracy.

Computer Vision and Pattern Recognition
Accelerated Verification of Autonomous Driving Systems Based on Adaptive Subset Simulation

Ye Tian,Aohui Fu,He Zhang,Lanyue Tang,Jian Sun

DOI: https://doi.org/10.1109/tiv.2024.3449947

IF: 8.2

2024-01-01

IEEE Transactions on Intelligent Vehicles

Abstract:A throughout safety verification of an Autonomous Driving System (ADS) is essential to ensure its capability to drive safely in naturalistic traffic environment. Commonly used test automation methods such as Important Sampling (IS) do not perform well in distribution fitting efficiency in high-dimensional scenarios. In this paper, we proposed the Adaptive Subset Simulation (ADSS) method for safety verification for ADS. Two different systems under test were adopted: Baidu Apollo and Intelligent Driver Model (IDM). ADSS finds scenarios that are more and more critical iteratively and is capable to derive the accurate collision rate in naturalistic driving environment with fewer testing resources. A 3-dimensional car-following scenario and a 6-dimensional cut-in scenario were constructed in LGSVL to evaluate the performance of ADSS. The results showed that as the number of scenario dimensions increases, ADSS exhibits a more significant advantage over IS and vanilla Subset Simulation (SS) in terms of result accuracy and acceleration efficiency. In the 6-dimensional cut-in scenario, the efficiency of ADSS was 112 times faster than that of IS and 2 times faster than that of SS. Notably, ADSS demonstrated excellent testing effectiveness for both the complex ADS Apollo and the simple ADS IDM. These findings highlight the significance of utilizing ADSS in the safety verification of ADS.
Adaptive-AR Model with Drivers' Prediction for Traffic Simulation

Xuequan Lu,Mingliang Xu,Wenzhi Chen,Zonghui Wang,Abdennour El Rhalibi

DOI: https://doi.org/10.1155/2013/904154

2013-01-01

International Journal of Computer Games Technology

Abstract:We present a novel model called A2R--"Adaptive-AR"--based on a well-known continuum-based model called AR Aw and Rascle (2000) for the simulation of vehicle traffic flows. However, in the standard continuum-based model, vehicles usually follow the flows passively, without taking into account drivers' behavior and effectiveness. In order to simulate real-life traffic flows, we extend the model with a few factors, which include the effectiveness of drivers' prediction, drivers' reaction time, and drivers' types. We demonstrate that our A2R model is effective and the results of the experiments agree well with experience in real world. It has been shown that such a model makes vehicle flows perform more realistically and is closer to the real-life traffic than AR (short for Aw and Rascle and introduced in Aw and Rascle (2000)) model while having a similar performance.
OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Driving

Guohang Yan,Jiahao Pi,Jianfei Guo,Zhaotong Luo,Min Dou,Nianchen Deng,Qiusheng Huang,Daocheng Fu,Licheng Wen,Pinlong Cai,Xing Gao,Xinyu Cai,Bo Zhang,Xuemeng Yang,Yeqi Bai,Hongbin Zhou,Botian Shi

2024-02-06

Abstract:With deep learning and computer vision technology development, autonomous driving provides new solutions to improve traffic safety and efficiency. The importance of building high-quality datasets is self-evident, especially with the rise of end-to-end autonomous driving algorithms in recent years. Data plays a core role in the algorithm closed-loop system. However, collecting real-world data is expensive, time-consuming, and unsafe. With the development of implicit rendering technology and in-depth research on using generative models to produce data at scale, we propose OASim, an open and adaptive simulator and autonomous driving data generator based on implicit neural rendering. It has the following characteristics: (1) High-quality scene reconstruction through neural implicit surface reconstruction technology. (2) Trajectory editing of the ego vehicle and participating vehicles. (3) Rich vehicle model library that can be freely selected and inserted into the scene. (4) Rich sensors model library where you can select specified sensors to generate data. (5) A highly customizable data generation system can generate data according to user needs. We demonstrate the high quality and fidelity of the generated data through perception performance evaluation on the Carla simulator and real-world data acquisition. Code is available at

Computer Vision and Pattern Recognition
A Simulation System for Human-in-the-Loop Driving

Yan Li,Yuanqi Su,Xiaoning Zhang,Qingchao Cai,Haoang Lu,Yuehu Liu

DOI: https://doi.org/10.1109/itsc55140.2022.9921924

2022-01-01

Abstract:By combining driver's ability with the machine intelligence, we are trying to augment the system with the combination of both roles during driving. The human-in-the-loop hybrid-augmented system is expected to monitor and model drivers' behavior; while the current datasets for driving mainly focus on the data of scene. The lack of the driver's data restricts the capability to understand driver's inner state. In the paper, we supply an effective way to monitor the driver and construct a simulation system for the human-in-the-loop autonomous driving, which realizes the modeling of driver's behavior. Based on it, we provide a data acquisition scheme for both driver's behavior and scene content and subsequently construct the corresponding dataset. For the simulated driving dataset, we provide a method for evaluating the human-in-the-loop-system, which can help the community to develop such human-in-the-loop systems.
Autonomous Driving Simulation for Unmanned Vehicles.

Danchen Zhao,Yuehu Liu,Chi Zhang,Yaochen Li

DOI: https://doi.org/10.1109/wacv.2015.32

2015-01-01

Abstract:Human can judge driver's driving ability by observing the vehicle motion in different traffic scenes. Identically, driving behavior can be the main basis for evaluating the performance of an unmanned vehicle in both field test and simulation test. Although simulation test avoids disadvantages of field test, existing simulation systems lack traffic scene data with perception granularity of visual sensors. In order for realizing vehicle-in-loop simulation, simulation technique of driving behaviors must be able to exhibit actual motion of unmanned vehicles. In this paper, we propose an automatic approach of simulating autonomous driving behaviors of vehicles in traffic scene represented by image sequences. Different from general simulation systems, we use actual traffic environment data to build the traffic scene and simulate the driving behaviors. After the proposed method was embedded in scene browser, a typical traffic scene including the intersections was chosen for virtual vehicle to execute the driving tasks of lane change, overtaking, slowing down and stop, right turn and U-Turn. The experimental results show that different driving behaviors of vehicles in typical traffic scene can be exhibited smoothly and realistically. Our method can also be used for generating simulation data of traffic scenes that are difficult to collect.
ADEPT: A Testing Platform for Simulated Autonomous Driving.

Sen Wang,Zhuheng Sheng,Jingwei Xu,Taolue Chen,Junjun Zhu,Shuhui Zhang,Yuan Yao,Xiaoxing Ma

DOI: https://doi.org/10.1145/3551349.3559528

2022-01-01

Abstract:Effective quality assurance methods for autonomous driving systems ADS have attracted growing interests recently. In this paper, we report a new testing platform ADEPT, aiming to provide practically realistic and comprehensive testing facilities for DNN-based ADS. ADEPT is based on the virtual simulator CARLA and provides numerous testing facilities such as scene construction, ADS importation, test execution and recording, etc. In particular, ADEPT features two distinguished test scenario generation strategies designed for autonomous driving. First, we make use of real-life accident reports from which we leverage natural language processing to fabricate abundant driving scenarios. Second, we synthesize physically-robust adversarial attacks by taking the feedback of ADS into consideration and thus are able to generate closed-loop test scenarios. The experiments confirm the efficacy of the platform.
Hybrid Operations of Human Driving Vehicles and Automated Vehicles with Data-Driven Agent-Based Simulation

Fugen Yao,Jiangtao Zhu,Jingru Yu,Chuqiao Chen,Xiqun (Michael) Chen

DOI: https://doi.org/10.1016/j.trd.2020.102469

IF: 7.041

2020-01-01

Transportation Research Part D Transport and Environment

Abstract:Automated vehicles (AVs) receive tremendous attention and achieve rapid development. It is foreseeable that hybrid operations of human driving vehicles and automated vehicles in the urban transportation environment will be a long-standing state. To investigate the influence of AVs on the hybrid ride-hailing market, data-driven agent-based modeling and simulation (D(2)ABMS) for large-scale transportation networks is proposed, in which human drivers, automated vehicles, and passengers form three types of agents. D(2)ABMS goes beyond existing approaches by employing data-driven multi-objective deep learning to learn ride-sourcing drivers' offline/online behavior. Embedding is used to represent the hidden attributes of different classes of drivers. Ride-sourcing data collected from the city of Hangzhou, China, are used to train and validate the drivers' decision-making model. Hybrid operations of human driving vehicles and automated vehicles with D(2)ABMS are comprehensively tested in various scenarios. The results show that a small proportion of automated vehicles in the hybrid ride-hailing market can significantly reduce the average waiting time of passengers. Besides, compared to the human driving scenario, the total exhaust emissions and vehicle kilometers traveled can be reduced by 12.3% in the AVs scenario. The proposed D(2)ABMS system has the potential to help transportation planners and ride-hailing platforms to assess their policies and operations management strategies in the era of shared mobility and automated vehicles.
Augmented Reality based Simulated Data (ARSim) with multi-view consistency for AV perception networks

Aqeel Anwar,Tae Eun Choe,Zian Wang,Sanja Fidler,Minwoo Park

2024-03-23

Abstract:Detecting a diverse range of objects under various driving scenarios is essential for the effectiveness of autonomous driving systems. However, the real-world data collected often lacks the necessary diversity presenting a long-tail distribution. Although synthetic data has been utilized to overcome this issue by generating virtual scenes, it faces hurdles such as a significant domain gap and the substantial efforts required from 3D artists to create realistic environments. To overcome these challenges, we present ARSim, a fully automated, comprehensive, modular framework designed to enhance real multi-view image data with 3D synthetic objects of interest. The proposed method integrates domain adaptation and randomization strategies to address covariate shift between real and simulated data by inferring essential domain attributes from real data and employing simulation-based randomization for other attributes. We construct a simplified virtual scene using real data and strategically place 3D synthetic assets within it. Illumination is achieved by estimating light distribution from multiple images capturing the surroundings of the vehicle. Camera parameters from real data are employed to render synthetic assets in each frame. The resulting augmented multi-view consistent dataset is used to train a multi-camera perception network for autonomous vehicles. Experimental results on various AV perception tasks demonstrate the superior performance of networks trained on the augmented dataset.

Computer Vision and Pattern Recognition,Machine Learning,Robotics
Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses

Minrui Xu,Dusit Niyato,Junlong Chen,Hongliang Zhang,Jiawen Kang,Zehui Xiong,Shiwen Mao,Zhu Han

DOI: https://doi.org/10.1109/jstsp.2023.3293650

IF: 7.695

2023-01-01

IEEE Journal of Selected Topics in Signal Processing

Abstract:In the vehicular mixed reality (MR) Metaverse, the discrepancy between physical and virtual entities can be overcome by fusing the physical and virtual environments with multi-dimensional communications in autonomous driving systems. Assisted by digital twin (DT) technologies, connected autonomous vehicles (AVs), roadside units (RSUs), and virtual simulators can maintain the vehicular MR Metaverse via simulations for sharing data and making driving decisions collaboratively. However, black it is challenging and costly to enable large-scale traffic and driving simulation via realistic data collection and fusion from the physical world for online prediction and offline training in autonomous driving systems. In this paper, we propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data black via simulations for improving driving safety and traffic black control efficiency. First, we propose a multi-task DT offloading model for the reliable execution of heterogeneous DT tasks with different requirements at RSUs. Then, based on the preferences of AV's DTs and black real-world data, virtual simulators can synthesize unlimited conditioned driving and traffic datasets black for improved robustness. Finally, we propose a multi-task enhanced auction-based mechanism to provide fine-grained incentives for RSUs on providing resources for autonomous driving. The property analysis and experimental results demonstrate that the proposed mechanism and architecture are strategy-proof and effective.

engineering, electrical & electronic
A Pioneering Scalable Self-driving Car Simulation Platform

Haolun Wu,Yunfei Feng

DOI: https://doi.org/10.1109/AUTEEE48671.2019.9033358

2019-01-01

Abstract:This paper proposes a novel hybrid, cross-platform 3D self-driving simulator, which offers a platform for researchers in the field of automatic vehicles to validate algorithms with ease and criteria to evaluate models in the community. Primarily our system decomposes the achievement of self-driving into four distinct steps by using a Tensorflow framework in machine learning: data generating, data balancing, model training, and model testing. The core part of our training section, the Alexnet, underlies training a deep neuron network with high accuracy at a fast speed. Some techniques, like balancing data, are employed to avoid overfilling, thus improving the robustness and accuracy; other functions, such as speed control, are designed for emulating the reality more vividly. By many study cases, it shows a strong flexibility of our simulator substantially. Various lane detection and obstacle detection algorithms are embedded smoothly in our system. In addition, Canny algorithm and Hough Transfer algorithm detect and draw out the edge of lanes perfectly, while YOLO architecture manages to detect obstacles.
Scalable Traffic Simulation for Autonomous Driving Via Multi-Agent Goal Assignment and Autoregressive Goal-Directed Planning

Xiaoyu Mo,Haochen Liu,Zhiyu Huang,Jianwu Fang,Jianru Xue,Chen Lv

DOI: https://doi.org/10.1109/iv55156.2024.10588681

2024-01-01

Abstract:Simulation provides a fast, cost-effective, and secure environment for developing autonomous driving systems. However, mitigating the gap between simulation and reality is a challenging task as it demands a behavior simulation method that is human-like, diverse, controllable, socially consistent, and scalable. This work proposes a data-driven traffic agent simulation method to address the aforementioned challenges. Our approach centers around a graph-based scene representation and an encoding method, dividing the simulation into two stages: Multi-Agent Goal assignment (MAG) and Goal-Directed Planning (GDP). Firstly, we create joint goal sets for all agents involved in the scenario. Subsequently, we assign target centerlines (TCLs) to each agent based on their predicted goals. To account for any potential mismatch between the predicted joint goal sets and the road structure, we further align the goals of each agent with their respective assigned TCLs. These on-TCL goals serve as inputs for our interactive autoregressive Goal-Directed Planner (AR-GDP), constituting the second stage of our method that generates roll-outs for simulations. Evaluation results on the leaderboard of the Waymo Open Sim Agents Challenge (WOSAC) 2023 show the competitiveness of the proposed method.
Data-driven Traffic Simulation: A Comprehensive Review

Di Chen,Meixin Zhu,Hao Yang,Xuesong Wang,Yinhai Wang

DOI: https://doi.org/10.1109/tiv.2024.3367919

IF: 8.2

2024-01-01

IEEE Transactions on Intelligent Vehicles

Abstract:Autonomous vehicles (AVs) have the potential to significantly revolutionize society by providing a secure and efficient mode of transportation. Recent years have witnessed notable advancements in autonomous driving perception and prediction, but the challenge of validating the performance of AVs remains largely unresolved. Data-driven microscopic traffic simulation has become an important tool for autonomous driving testing due to 1) availability of high-fidelity traffic data; 2) its advantages of enabling large-scale testing and scenario reproducibility; and 3) its potential in reactive and realistic traffic simulation. However, a comprehensive review of this topic is currently lacking. This paper aims to fill this gap by summarizing relevant studies. The primary objective of this paper is to review current research efforts and provide a futuristic perspective that will benefit future developments in the field. It introduces the general issues of data-driven traffic simulation and outlines key concepts and terms. After overviewing traffic simulation, various datasets and evaluation metrics commonly used are reviewed. The paper then offers a comprehensive evaluation of imitation learning, reinforcement learning, deep generative and deep learning methods, summarizing each and analyzing their advantages and disadvantages in detail. Moreover, it evaluates the state-of-the-art, existing challenges, and future research directions.

English Else
A Digital Twin-based Testing and Data Collection System for Autonomous Driving in Extreme Traffic Scenarios

Yueran Ding,Junhong Zou,Yixuan Fan,Shengjin Wang,Qingmin Liao

DOI: https://doi.org/10.1145/3579109.3579127

2022-01-01

Abstract:Autonomous driving systems need to undergo rigorous testing in complex scenarios including a variety of extreme operating conditions before they can be put into use. In this process, digital twin technology can migrate the scenes in the physical world to computer simulation software, so that engineers can comprehensively and safely conduct large-scale test experiments, get well prepared for vehicle tests on real road. The existing related systems in academia are still lacking in the authenticity, diversity and complexity of the scene. To solve the above problems, we propose a digital twin-based test and data acquisition system for autonomous driving under extreme traffic scenes. Based on UE4 engine and CARLA simulation platform, the system includes a Chinese urban style map with a total area of 10.8 square kilometers and 232 multi-site test routes including 29 possible events, 4 sets of environmental parameters, 2 sets of location parameters, and all weather condition. At the same time, a large amount of sensor data can also be collected in the system, which fills the insufficiency of collecting extreme working scene data in the real world. We give a perception-oriented autonomous driving data collection scheme, which can store sensor output while running simulation events, and automatically generate corresponding annotations, which can collect data in emergency situations. We also propose a new evaluation metric of autonomous driving system based on research on accident hazard. This test system covers a variety of extreme working conditions that are not involved in the existing systems, and puts forward higher requirements for the perception and decision-making algorithms related to autonomous driving.
Real-World Data Inspired Interactive Connected Traffic Scenario Generation

Junwei You,Pei Li,Yang Cheng,Keshu Wu,Rui Gan,Steven T. Parker,Bin Ran

2024-09-26

Abstract:Simulation is a crucial step in ensuring accurate, efficient, and realistic Connected and Autonomous Vehicles (CAVs) testing and validation. As the adoption of CAV accelerates, the integration of real-world data into simulation environments becomes increasingly critical. Among various technologies utilized by CAVs, Vehicle-to-Everything (V2X) communication plays a crucial role in ensuring a seamless transmission of information between CAVs, infrastructure, and other road users. However, most existing studies have focused on developing and testing communication protocols, resource allocation strategies, and data dissemination techniques in V2X. There is a gap where real-world V2X data is integrated into simulations to generate diverse and high-fidelity traffic scenarios. To fulfill this research gap, we leverage real-world Signal Phase and Timing (SPaT) data from Roadside Units (RSUs) to enhance the fidelity of CAV simulations. Moreover, we developed an algorithm that enables Autonomous Vehicles (AVs) to respond dynamically to real-time traffic signal data, simulating realistic V2X communication scenarios. Such high-fidelity simulation environments can generate multimodal data, including trajectory, semantic camera, depth camera, and bird's eye view data for various traffic scenarios. The generated scenarios and data provide invaluable insights into AVs' interactions with traffic infrastructure and other road users. This work aims to bridge the gap between theoretical research and practical deployment of CAVs, facilitating the development of smarter and safer transportation systems.

Robotics
LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving

Guodong Rong,Byung Hyun Shin,Hadi Tabatabaee,Qiang Lu,Steve Lemke,Mārtiņš Možeiko,Eric Boise,Geehoon Uhm,Mark Gerow,Shalin Mehta,Eugene Agafonov,Tae Hyung Kim,Eric Sterner,Keunhae Ushiroda,Michael Reyes,Dmitry Zelenkovsky,Seonman Kim

2020-06-22

Abstract:Testing autonomous driving algorithms on real autonomous vehicles is extremely costly and many researchers and developers in the field cannot afford a real car and the corresponding sensors. Although several free and open-source autonomous driving stacks, such as Autoware and Apollo are available, choices of open-source simulators to use with them are limited. In this paper, we introduce the LGSVL Simulator which is a high fidelity simulator for autonomous driving. The simulator engine provides end-to-end, full-stack simulation which is ready to be hooked up to Autoware and Apollo. In addition, simulator tools are provided with the core simulation engine which allow users to easily customize sensors, create new types of controllable objects, replace some modules in the core simulator, and create digital twins of particular environments.

Robotics,Machine Learning,Systems and Control
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking

Daniel Dauner,Marcel Hallgarten,Tianyu Li,Xinshuo Weng,Zhiyu Huang,Zetong Yang,Hongyang Li,Igor Gilitschenski,Boris Ivanovic,Marco Pavone,Andreas Geiger,Kashyap Chitta

2024-11-01

Abstract:Benchmarking vision-based driving policies is challenging. On one hand, open-loop evaluation with real data is easy, but these results do not reflect closed-loop performance. On the other, closed-loop evaluation is possible in simulation, but is hard to scale due to its significant computational demands. Further, the simulators available today exhibit a large domain gap to real data. This has resulted in an inability to draw clear conclusions from the rapidly growing body of research on end-to-end autonomous driving. In this paper, we present NAVSIM, a middle ground between these evaluation paradigms, where we use large datasets in combination with a non-reactive simulator to enable large-scale real-world benchmarking. Specifically, we gather simulation-based metrics, such as progress and time to collision, by unrolling bird's eye view abstractions of the test scenes for a short simulation horizon. Our simulation is non-reactive, i.e., the evaluated policy and environment do not influence each other. As we demonstrate empirically, this decoupling allows open-loop metric computation while being better aligned with closed-loop evaluations than traditional displacement errors. NAVSIM enabled a new competition held at CVPR 2024, where 143 teams submitted 463 entries, resulting in several new insights. On a large set of challenging scenarios, we observe that simple methods with moderate compute requirements such as TransFuser can match recent large-scale end-to-end driving architectures such as UniAD. Our modular framework can potentially be extended with new datasets, data curation strategies, and metrics, and will be continually maintained to host future challenges. Our code is available at <a class="link-external link-https" href="https://github.com/autonomousvision/navsim" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning,Robotics
SCTrans: Constructing a Large Public Scenario Dataset for Simulation Testing of Autonomous Driving Systems.

Jiarun Dai,Bufan Gao,Mingyuan Luo,Zongan Huang,Zhongrui Li,Yuan Zhang,Min Yang

DOI: https://doi.org/10.1145/3597503.3623350

2024-01-01

Abstract:For the safety assessment of autonomous driving systems (ADS), simulation testing has become an important complementary technique to physical road testing. In essence, simulation testing is a scenario-driven approach, whose effectiveness is highly dependent on the quality of given simulation scenarios. Moreover, simulation scenarios should be encoded into well-formatted files, otherwise, ADS simulation platforms cannot take them as inputs. Without large public datasets of simulation scenario files, both industry and academic applications of ADS simulation testing are hindered. To fill this gap, we propose a transformation-based approach SCTRANS to construct simulation scenario files, utilizing existing traffic scenario datasets (i.e., naturalistic movement of road users recorded on public roads) as data sources. Specifically, we try to transform existing traffic scenario recording files into simulation scenario files that are compatible with the most advanced ADS simulation platforms, and this task is formalized as a Model Transformation Problem. Following this idea, we construct a dataset consisting of over 1,900 diverse simulation scenarios, each of which can be directly used to test the state-of-the-art ADSs (i.e., Apollo and Autoware) via high-fidelity simulators (i.e., Carla and LGSVL). To further demonstrate the utility of our dataset, we showcase that it can boost the collision-finding capability of existing simulation-based ADS fuzzers, helping identify about seven times more unique ADS-involved collisions within the same time period. By analyzing these collisions at the code level, we identify nine safety-critical bugs of Apollo and Autoware, each of which can be stably exploited to cause vehicle crashes. Till now, four of them have been confirmed.

AADS: Augmented autonomous driving simulation using data-driven algorithms

AADS: Augmented autonomous driving simulation using data-driven algorithms

Augmented LiDAR Simulator for Autonomous Driving

Accelerated Verification of Autonomous Driving Systems Based on Adaptive Subset Simulation

Adaptive-AR Model with Drivers' Prediction for Traffic Simulation

OASim: an Open and Adaptive Simulator based on Neural Rendering for Autonomous Driving

A Simulation System for Human-in-the-Loop Driving

Autonomous Driving Simulation for Unmanned Vehicles.

ADEPT: A Testing Platform for Simulated Autonomous Driving.

Hybrid Operations of Human Driving Vehicles and Automated Vehicles with Data-Driven Agent-Based Simulation

Augmented Reality based Simulated Data (ARSim) with multi-view consistency for AV perception networks

Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses

A Pioneering Scalable Self-driving Car Simulation Platform

Scalable Traffic Simulation for Autonomous Driving Via Multi-Agent Goal Assignment and Autoregressive Goal-Directed Planning

Data-driven Traffic Simulation: A Comprehensive Review

A Digital Twin-based Testing and Data Collection System for Autonomous Driving in Extreme Traffic Scenarios

Real-World Data Inspired Interactive Connected Traffic Scenario Generation

LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving

NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking

SCTrans: Constructing a Large Public Scenario Dataset for Simulation Testing of Autonomous Driving Systems.