Abstract:Evaluating the decision-making system is indispensable in developing autonomous vehicles, while realistic and challenging safety-critical test scenarios play a crucial role. Obtaining these scenarios is non-trivial, thanks to the long-tailed distribution, sparsity, and rarity in real-world data sets. To tackle this problem, in this paper, we introduce a natural adversarial scenario generation solution using naturalistic human driving priors and reinforcement learning techniques. By doing this, we can obtain large-scale test scenarios that are both diverse and realistic. Specifically, we build a simulation environment that mimics natural traffic interaction scenarios. Informed by this environment, we implement a two-stage procedure. The first stage incorporates conventional rule-based models, e.g., IDM~(Intelligent Driver Model) and MOBIL~(Minimizing Overall Braking Induced by Lane changes) model, to coarsely and discretely capture and calibrate key control parameters from the real-world dataset. Next, we leverage GAIL~(Generative Adversarial Imitation Learning) to represent driver behaviors continuously. The derived GAIL can be further used to design a PPO~(Proximal Policy Optimization)-based actor-critic network framework to fine-tune the reward function, and then optimizes our natural adversarial scenario generation solution. Extensive experiments have been conducted in the NGSIM dataset including the trajectory of 3,000 vehicles. Essential traffic parameters were measured in comparison with the baseline model, e.g., the collision rate, accelerations, steering, and the number of lane changes. Our findings demonstrate that the proposed model can generate realistic safety-critical test scenarios covering both naturalness and adversariality, which can be a cornerstone for the development of autonomous vehicles.

Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge

Continuous Decision-Making in Lane Changing and Overtaking Maneuvers for Unmanned Vehicles: A Risk-Aware Reinforcement Learning Approach with Task Decomposition

CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening

Multi-Vehicle Mixed-Reality Reinforcement Learning for Autonomous Multi-Lane Driving

Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning

Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors

Don't Get Yourself into Trouble! Risk-aware Decision-Making for Autonomous Vehicles

Game-Theoretic Planning for Autonomous Driving among Risk-Aware Human Drivers

Behavior Identification and Prediction for a Probabilistic Risk Framework

Autonomous Highway Driving using Deep Reinforcement Learning

Adversarial Driving Behavior Generation Incorporating Human Risk Cognition for Autonomous Vehicle Evaluation

Learning Naturalistic Driving Environment with Statistical Realism

Multi-Agent Vulnerability Discovery for Autonomous Driving with Hazard Arbitration Reward

Adversarial and Reactive Traffic Agents for Realistic Driving Simulation

Enhancing Autonomous Lane-Changing Safety: Deep Reinforcement Learning Via Pre-Exploration in Parallel Imaginary Environments

Microscopic Traffic Simulation by Cooperative Multi-agent Deep Reinforcement Learning

Adversarial Generation of Safety-Critical Lane-Change Scenarios for Autonomous Vehicles

Deep Reinforcement Learning for Human-Like Driving Policies in Collision Avoidance Tasks of Self-Driving Cars

Rare Collision Risk Estimation of Autonomous Vehicles with Multi-Agent Situation Awareness

Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving

Learning Interactive Driving Policies via Data-driven Simulation