Abstract:Autonomous driving systems (ADSs) are capable of sensing the environment and making driving decisions autonomously. These systems are safety-critical, and testing them is one of the important approaches to ensure their safety. However, due to the inherent complexity of ADSs and the high dimensionality of their operating environment, the number of possible test scenarios for ADSs is infinite. Besides, the operating environment of ADSs is dynamic, continuously evolving, and full of uncertainties, which requires a testing approach adaptive to the environment. In addition, existing ADS testing techniques have limited effectiveness in ensuring the realism of test scenarios, especially the realism of weather conditions and their changes over time. Recently, reinforcement learning (RL) has demonstrated great potential in addressing challenging problems, especially those requiring constant adaptations to dynamic environments. To this end, we present DeepQTest, a novel ADS testing approach that uses RL to learn environment configurations with a high chance of revealing abnormal ADS behaviors. Specifically, DeepQTest employs Deep Q-Learning and adopts three safety and comfort measures to construct the reward functions. To ensure the realism of generated scenarios, DeepQTest defines a set of realistic constraints and introduces real-world weather conditions into the simulated environment. We employed three comparison baselines, i.e., random, greedy, and a state-of-the-art RL-based approach DeepCOllision, for evaluating DeepQTest on an industrial-scale ADS. Evaluation results show that DeepQTest demonstrated significantly better effectiveness in terms of generating scenarios leading to collisions and ensuring scenario realism compared with the baselines. In addition, among the three reward functions implemented in DeepQTest, Time-To-Collision is recommended as the best design according to our study.

Test Scenario Generation for Autonomous Driving Systems with Reinforcement Learning

DeepQTest: Testing Autonomous Driving Systems with Reinforcement Learning and Real-world Weather Data

Generating Critical Scenarios for Testing Automated Driving Systems

Critical concrete scenario generation using scenario-based falsification

Adaptive Deep Reinforcement Learning for Critical Boundary Scenario Generation

ISS-Scenario: Scenario-based Testing in CARLA

Towards Automated Safety Coverage and Testing for Autonomous Vehicles with Reinforcement Learning

DeepScenario: An Open Driving Scenario Dataset for Autonomous Driving System Testing

Restricted Natural Language and Model-based Adaptive Test Generation for Autonomous Driving

Adversarial Generation of Safety-Critical Lane-Change Scenarios for Autonomous Vehicles

Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study

A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation

Realistic Safety-critical Scenarios Search for Autonomous Driving System via Behavior Tree

Reinforcement Learning for Safety Testing: Lessons from A Mobile Robot Case Study

Adaptive Testing Environment Generation for Connected and Automated Vehicles with Dense Reinforcement Learning

Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize Their Collisions

A Real-time Critical-scenario-generation Framework for Testing Autonomous Driving System

SCTrans: Constructing a Large Public Scenario Dataset for Simulation Testing of Autonomous Driving Systems.

CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening

Deep Reinforcement Learning for Adverse Garage Scenario Generation

Collision Avoidance Testing for Autonomous Driving Systems on Complete Maps