Abstract:The wide availability of data coupled with the computational advances in artificial intelligence and machine learning promise to enable many future technologies such as autonomous driving. While there has been a variety of successful demonstrations of these technologies, critical system failures have repeatedly been reported. Even if rare, such system failures pose a serious barrier to adoption without a rigorous risk assessment. This paper presents a framework for the systematic and rigorous risk verification of systems. We consider a wide range of system specifications formulated in signal temporal logic (STL) and model the system as a stochastic process, permitting discrete-time and continuous-time stochastic processes. We then define the STL robustness risk as the risk of lacking robustness against failure. This definition is motivated as system failures are often caused by missing robustness to modeling errors, system disturbances, and distribution shifts in the underlying data generating process. Within the definition, we permit general classes of risk measures and focus on tail risk measures such as the value-at-risk and the conditional value-at-risk. While the STL robustness risk is in general hard to compute, we propose the approximate STL robustness risk as a more tractable notion that upper bounds the STL robustness risk. We show how the approximate STL robustness risk can accurately be estimated from system trajectory data. For discrete-time stochastic processes, we show under which conditions the approximate STL robustness risk can even be computed exactly. We illustrate our verification algorithm in the autonomous driving simulator CARLA and show how a least risky controller can be selected among four neural network lane keeping controllers for five meaningful system specifications.

Stochastic Assume-Guarantee Contracts for Cyber-Physical System Design

Stabilization of Networked Control Systems with Hybrid-Driven Mechanism and Probabilistic Cyber Attacks.

Verifying the Unknown: Correct-by-Design Control Synthesis for Networks of Stochastic Uncertain Systems

A Control Approach for Nonlinear Stochastic State Uncertain Systems with Probabilistic Safety Guarantees

Probabilistic Tube-based Control Synthesis of Stochastic Multi-Agent Systems under Signal Temporal Logic

Risk of Stochastic Systems for Temporal Logic Specifications

Formal Synthesis of Stochastic Systems via Control Barrier Certificates

Correct-by-Construction Control for Stochastic and Uncertain Dynamical Models via Formal Abstractions

Controller Synthesis of Collaborative Signal Temporal Logic Tasks for Multi-Agent Systems via Assume-Guarantee Contracts

Adaptive Event-Driven Control for Continuous Positive Stochastic Jump Models with Cyber Attacks

Stochastic Lyapunov-Barrier Functions for Robust Probabilistic Reach-Avoid-Stay Specifications

Robust Dynamic Programming for Temporal Logic Control of Stochastic Systems

Control Design for Risk-Based Signal Temporal Logic Specifications

From Small-Gain Theory to Compositional Construction of Barrier Certificates for Large-Scale Stochastic Systems

Secure-by-Construction Controller Synthesis for Stochastic Systems under Linear Temporal Logic Specifications

Myopically Verifiable Probabilistic Certificates for Safe Control and Learning

Probabilistic Guarantees for Nonlinear Safety-Critical Optimal Control

Uncertainty-Aware Reactive Control for Signal Temporal Logic with Unknown Obstacles

Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics

Conformance Testing for Stochastic Cyber-Physical Systems

Tunable Reactive Synthesis for Lipschitz-Bounded Systems with Temporal Logic Specifications