Abstract:Fully autonomous driving is considered one of the grand challenges of modern technology and a variety of approaches have emerged for creating and evaluating autonomous driving agents. The self-driving industry typically adopts a modular software architecture and uses large fleets of autonomous vehicles for data collection and evaluation. Simulation generally occupies a minor role in these workflows as a tool for basic unit tests and debugging. However, recent end-to-end methods in academia have elevated simulation’s role in the training of machine learning models and demonstrate state-of-the-art performance on simulated driving benchmarks as a result. In this thesis, we consider two questions. First, what are the state-ofthe-art approaches for end-to-end driving in simulation and how can we improve them? We use the open-source CARLA simulator as our testbed and work with the recent Leaderboard benchmark for evaluation. In our first project, we give a detailed failure analysis of “Learning by Cheating” (LBC) which has been considered the state-of-the-art in imitation learning. We then propose offline imitative reinforcement learning (OIRL) as an alternative approach and investigate its performance. In our second project, we give a failure analysis of the “World on Rails” (WOR) offline reinforcement learning method and propose “Slightly Online Reinforcement Learning” (SORL) as an improvement. Second, what value does simulation provide for the training and evaluation of driving algorithms? We draw attention to the crucial role that privileged information plays in both LBC and WOR’s algorithms. We emphasize that these methods are impossible to replicate without simulation and advocate for privileged information as an important tool for developing planning algorithms. Finally, we give a critical analysis of CARLA by pointing out issues in its perceptual and behavioral realism. We also critique the Leaderboard benchmark and provide discussion on why its primary metric cannot be reliably used as a proxy for good driving behavior.

From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Driving

From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Driving

Exiting the Simulation: The Road to Robust and Resilient Autonomous Vehicles at Scale

Challenges of Data-Driven Simulation of Diverse and Consistent Human Driving Behaviors

An Approach to Systematic Data Acquisition and Data-Driven Simulation for the Safety Testing of Automated Driving Functions

Data-driven Traffic Simulation: A Comprehensive Review

The State of Modeling, Simulation, and Data Utilization within Industry: An Autonomous Vehicles Perspective

How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence

End-to-end Methods for Autonomous Driving in Simulation

Learning Interactive Driving Policies via Data-driven Simulation

A Joint Approach Towards Data-Driven Virtual Testing for Automated Driving: The AVEAS Projec

Data Generation Using Simulation Technology to Improve Perception Mechanism of Autonomous Vehicles

Virtual Tools for Testing Autonomous Driving: A Survey and Benchmark of Simulators, Datasets, and Competitions

Diagnosing and Predicting Autonomous Vehicle Operational Safety Using Multiple Simulation Modalities and a Virtual Environment

A Systematic Solution of Human Driving Behavior Modeling and Simulation for Automated Vehicle Studies

Spatio-Semantic Road Space Modeling for Vehicle–Pedestrian Simulation to Test Automated Driving Systems

A Concept For A Virtual Reality Driving Simulation In Combination With A Real Car

Simulating Dynamic Driving Behavior in Simulation Test for Unmanned Vehicles via Multi-Sensor Data.

Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real World

Investigating Driving Interactions: A Robust Multi-Agent Simulation Framework for Autonomous Vehicles

Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation