Abstract:Context: Demonstrating high reliability and safety for safety-critical systems (SCSs) remains a hard problem. Diverse evidence needs to be combined in a rigorous way: in particular, results of operational testing with other evidence from design and verification. Growing use of machine learning in SCSs, by precluding most established methods for gaining assurance, makes evidence from operational testing even more important for supporting safety and reliability claims.Objective: We revisit the problem of using operational testing to demonstrate high reliability. We use Autonomous Vehicles (AVs) as a current example. AVs are making their debut on public roads: methods for assessing whether an AV is safe enough are urgently needed. We demonstrate how to answer 5 questions that would arise in assessing an AV type, starting with those proposed by a highly-cited study.Method: We apply new theorems extending our Conservative Bayesian Inference (CBI) approach, which exploit the rigour of Bayesian methods while reducing the risk of involuntary misuse associated (we argue) with now-common applications of Bayesian inference; we define additional conditions needed for applying these methods to AVs.Results: Prior knowledge can bring substantial advantages if the AV design allows strong expectations of safety before road testing. We also show how naive attempts at conservative assessment may lead to over-optimism instead; why extrapolating the trend of disengagements (take-overs by human drivers) is not suitable for safety claims; use of knowledge that an AV has moved to a "less stressful" environment.Conclusion: While some reliability targets will remain too high to be practically verifiable, our CBI approach removes a major source of doubt: it allows use of prior knowledge without inducing dangerously optimistic biases. For certain ranges of required reliability and prior beliefs, CBI thus supports feasible, sound arguments. Useful conservative claims can be derived from limited prior knowledge.

Learning Autonomous Vehicle Safety Concepts from Demonstrations

Road to safe autonomy with data and formal reasoning

Safety of autonomous vehicles: A survey on Model-based vs. AI-based approaches

Autonomous Highway Driving using Deep Reinforcement Learning

Towards Automated Safety Coverage and Testing for Autonomous Vehicles with Reinforcement Learning

Learning Responsibility Allocations for Safe Human-Robot Interaction with Applications to Autonomous Driving

Making autonomous vehicle systems human-like: lessons learned from accident experiences in traffic

Safe by Design Autonomous Driving Systems

A Data-Informed Analysis of Scalable Supervision for Safety in Autonomous Vehicle Fleets

Towards common ethical and safe 'behaviour' standards for automated vehicles

Safe Imitation Learning on Real-Life Highway Data for Human-like Autonomous Driving

Simulation-based Safety Assurance for an AVP System incorporating Learning-Enabled Components

Safety and Risk Analysis of Autonomous Vehicles Using Computer Vision and Neural Networks

Diagnosing and Predicting Autonomous Vehicle Operational Safety Using Multiple Simulation Modalities and a Virtual Environment

Automated and Complete Generation of Traffic Scenarios at Road Junctions Using a Multi-level Danger Definition

Safe, Efficient and Socially-Compatible Decision of Automated Vehicles: a Case Study of Unsignalized Intersection Driving

Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing

Assessing safety-critical systems from operational testing: A study on autonomous vehicles

Learning Driver Models for Automated Vehicles via Knowledge Sharing and Personalization

Verisimilar Percept Sequences Tests for Autonomous Driving Intelligent Agent Assessment

Efficient falsification approach for autonomous vehicle validation using a parameter optimisation technique based on reinforcement learning