Abstract:Autonomous vehicles (AVs) have demonstrated significant potential in revolutionizing transportation, yet ensuring their safety and reliability remains a critical challenge, especially when exposed to dynamic and unpredictable environments. Real-world testing of an Autonomous Driving System (ADS) is both expensive and risky, making simulation-based testing a preferred approach. In this paper, we propose AVASTRA, a Reinforcement Learning (RL)-based approach to generate realistic critical scenarios for testing ADSs in simulation environments. To capture the complexity of driving scenarios, AVASTRA comprehensively represents the environment by both the internal states of an ADS under-test (e.g., the status of the ADS's core components, speed, or acceleration) and the external states of the surrounding factors in the simulation environment (e.g., weather, traffic flow, or road condition). AVASTRA trains the RL agent to effectively configure the simulation environment that places the AV in dangerous situations and potentially leads it to collisions. We introduce a diverse set of actions that allows the RL agent to systematically configure both environmental conditions and traffic participants. Additionally, based on established safety requirements, we enforce heuristic constraints to ensure the realism and relevance of the generated test scenarios. AVASTRA is evaluated on two popular simulation maps with four different road configurations. Our results show AVASTRA's ability to outperform the state-of-the-art approach by generating 30% to 115% more collision scenarios. Compared to the baseline based on Random Search, AVASTRA achieves up to 275% better performance. These results highlight the effectiveness of AVASTRA in enhancing the safety testing of AVs through realistic comprehensive critical scenario generation.

Adversarial Stress Test for Autonomous Vehicle Via Series Reinforcement Learning Tasks with Reward Shaping

A novel framework for adaptive stress testing of autonomous vehicles in multi-lane roads

CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening

Continuous Decision-Making in Lane Changing and Overtaking Maneuvers for Unmanned Vehicles: A Risk-Aware Reinforcement Learning Approach with Task Decomposition

Multi-Agent Vulnerability Discovery for Autonomous Driving with Hazard Arbitration Reward

Adversarial Driving Behavior Generation Incorporating Human Risk Cognition for Autonomous Vehicle Evaluation

Potential-based Credit Assignment for Cooperative RL-based Testing of Autonomous Vehicles

VCAT: Vulnerability-aware and Curiosity-driven Adversarial Training for Enhancing Autonomous Vehicle Robustness

Adaptive Stress Testing for Autonomous Vehicles

Trustworthy autonomous driving via defense-aware robust reinforcement learning against worst-case observational perturbations

Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving

Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learning

Dense reinforcement learning for safety validation of autonomous vehicles

Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies

Towards Automated Safety Coverage and Testing for Autonomous Vehicles with Reinforcement Learning

Adversarial Generation of Safety-Critical Lane-Change Scenarios for Autonomous Vehicles

Efficient Autonomy Validation in Simulation with Adaptive Stress Testing

Generating Critical Scenarios for Testing Automated Driving Systems

Adaptive Testing Environment Generation for Connected and Automated Vehicles with Dense Reinforcement Learning

Risk Averse Robust Adversarial Reinforcement Learning

Enhancing Autonomous Lane-Changing Safety: Deep Reinforcement Learning Via Pre-Exploration in Parallel Imaginary Environments