Abstract:Fully autonomous driving is considered one of the grand challenges of modern technology and a variety of approaches have emerged for creating and evaluating autonomous driving agents. The self-driving industry typically adopts a modular software architecture and uses large fleets of autonomous vehicles for data collection and evaluation. Simulation generally occupies a minor role in these workflows as a tool for basic unit tests and debugging. However, recent end-to-end methods in academia have elevated simulation’s role in the training of machine learning models and demonstrate state-of-the-art performance on simulated driving benchmarks as a result. In this thesis, we consider two questions. First, what are the state-ofthe-art approaches for end-to-end driving in simulation and how can we improve them? We use the open-source CARLA simulator as our testbed and work with the recent Leaderboard benchmark for evaluation. In our first project, we give a detailed failure analysis of “Learning by Cheating” (LBC) which has been considered the state-of-the-art in imitation learning. We then propose offline imitative reinforcement learning (OIRL) as an alternative approach and investigate its performance. In our second project, we give a failure analysis of the “World on Rails” (WOR) offline reinforcement learning method and propose “Slightly Online Reinforcement Learning” (SORL) as an improvement. Second, what value does simulation provide for the training and evaluation of driving algorithms? We draw attention to the crucial role that privileged information plays in both LBC and WOR’s algorithms. We emphasize that these methods are impossible to replicate without simulation and advocate for privileged information as an important tool for developing planning algorithms. Finally, we give a critical analysis of CARLA by pointing out issues in its perceptual and behavioral realism. We also critique the Leaderboard benchmark and provide discussion on why its primary metric cannot be reliably used as a proxy for good driving behavior.

CARLA: An Open Urban Driving Simulator

CARLA Real Traffic Scenarios -- novel training ground and benchmark for autonomous driving

ANTI-CARLA: An Adversarial Testing Framework for Autonomous Vehicles in CARLA

CARLA Connect: A Connected Autonomous Vehicle (CAV) Driving Simulator

CARLOS: An Open, Modular, and Scalable Simulation Framework for the Development and Testing of Software for C-ITS

Train here, drive there: ROS based end-to-end autonomous-driving pipeline validation in CARLA simulator using the NHTSA typology

Deep Learning based Object Detection Model for Autonomous Driving Research using CARLA Simulator

A Novel Traffic Simulation Framework for Testing Autonomous Vehicles Using SUMO and CARLA

CARLA-GymDrive: Autonomous driving episode generation for the Carla simulator in a gym environment

A Software Architecture for Autonomous Vehicles: Team LRM-B Entry in the First CARLA Autonomous Driving Challenge

End-to-end Methods for Autonomous Driving in Simulation

CARLA-Autoware-Bridge: Facilitating Autonomous Driving Research with a Unified Framework for Simulation and Module Development

Enhancing the Realism of Autonomous Driving Simulation with Real-Time Co-Simulation.

Safe Navigation: Training Autonomous Vehicles using Deep Reinforcement Learning in CARLA

Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)

Think2Drive: Efficient Reinforcement Learning by Thinking with Latent World Model for Autonomous Driving (in CARLA-v2)

Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator

SLAV-Sim: A Framework for Self-Learning Autonomous Vehicle Simulation

CARLA2Real: a tool for reducing the sim2real gap in CARLA simulator

CARLA-Loc: Synthetic SLAM Dataset with Full-stack Sensor Setup in Challenging Weather and Dynamic Environments

CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving