Abstract:Real-time safety metrics are important for the automated driving system (ADS) to assess the risk of driving situations and to assist the decision-making. Although a number of real-time safety metrics have been proposed in the literature, systematic performance evaluation of these safety metrics has been lacking. As different behavioral assumptions are adopted in different safety metrics, it is difficult to compare the safety metrics and evaluate their performance. To overcome this challenge, in this study, we propose an evaluation framework utilizing logged vehicle trajectory data, in that vehicle trajectories for both subject vehicle (SV) and background vehicles (BVs) are obtained and the prediction errors caused by behavioral assumptions can be eliminated. Specifically, we examine whether the SV is in a collision unavoidable situation at each moment, given all near-future trajectories of BVs. In this way, we level the ground for a fair comparison of different safety metrics, as a good safety metric should always alarm in advance to the collision unavoidable moment. When trajectory data from a large number of trips are available, we can systematically evaluate and compare different metrics' statistical performance. In the case study, three representative real-time safety metrics, including the time-to-collision (TTC), the PEGASUS Criticality Metric (PCM), and the Model Predictive Instantaneous Safety Metric (MPrISM), are evaluated using a large-scale simulated trajectory dataset. The proposed evaluation framework is important for researchers, practitioners, and regulators to characterize different metrics, and to select appropriate metrics for different applications. Moreover, by conducting failure analysis on moments when a safety metric failed, we can identify its potential weaknesses which are valuable for its potential refinements and improvements.

The Comfort of the Soft-Safety Driver Alerts: Measurements and Evaluation

Safety Evaluation for Expressways: A Comparative Study for Macroscopic and Microscopic Indicators

Evaluating the Efficacy of Real-Time Connected Vehicle Basic Safety Messages in Mitigating Aberrant Driving Behaviour and Risk of Vehicle Crashes: Preliminary Insights from Highway Scenarios

Study on the Influence of Connected Vehicle Fog Warning Systems on Driving Behavior and Safety

Driving Confidence in a Connected Vehicle Environment: A Case Study of Expressway Work Zone

Effectiveness Assessment of Human-Machine Interface in Driver Assistance System

A Method for Vehicle Collision Risk Assessment through Inferring Driver's Braking Actions in Near-Crash Situations

Driver operational level identification of driving risk and graded time-based alarm under near-crash conditions: A driving simulator study

Evaluation of Infrastructure-based Warning System on Driving Behaviors-A Roundabout Study

Evaluation of automated driving system safety metrics with logged vehicle trajectory data

A Quantitative Method of Assessing the Hazard Perception of Drivers in Human–Machine Codriving Vehicles

Evaluation of Forward Collision Avoidance System Using Driver's Hazard Perception

Beyond the Dashboard: Investigating Distracted Driver Communication Preferences for ADAS

Research on the effectiveness of a intersection risk warning system based on driving simulation experiment

Latent Hazard Notification for Highly Automated Driving: Expected Safety Benefits and Driver Behavioral Adaptation

Long-Term Evaluation of Drivers’ Behavioral Adaptation to an Adaptive Collision Avoidance System

Comprehensive Evaluation of Freeway Driving Risks Based on Fuzzy Logic

Road Hazard Reaction Testing Using Driving Simulation: the Novice Vs. the Experienced Drivers

Objective metrics of comfort: Developing a driving style for highly automated vehicles

Simulator evaluation of an intersection maneuver assist system with connected and automated vehicle technologies