Abstract:While neural networks (NNs) have potential as autonomous controllers for Cyber-Physical Systems, verifying the safety of NN based control systems (NNCSs) poses significant challenges for the practical use of NNs, especially when safety is needed for unbounded time horizons. One reason is the intractability of analyzing NNs, ODEs and hybrid systems. To this end, we introduce VerSAILLE (Verifiably Safe AI via Logically Linked Envelopes): The first general approach that allows reusing control theory results for NNCS verification. By joining forces, we exploit the efficiency of NN verification tools while retaining the rigor of differential dynamic logic (dL). Based on provably safe control envelopes in dL, we derive specifications for the NN which is proven via NN verification. We show that a proof of the NN adhering to the specification is mirrored by a dL proof on the infinite-time safety of the NNCS. The NN verification properties resulting from hybrid systems typically contain nonlinear arithmetic and arbitrary logical structures while efficient NN verification merely supports linear constraints. To overcome this divide, we present Mosaic: An efficient, sound and complete verification approach for polynomial real arithmetic properties on piece-wise linear NNs. Mosaic partitions complex verification queries into simple queries and lifts off-the-shelf linear constraint tools to the nonlinear setting in a completeness-preserving manner by combining approximation with exact reasoning for counterexample regions. Our evaluation demonstrates the versatility of VerSAILLE and Mosaic: We prove infinite-time safety on the classical Vertical Airborne Collision Avoidance NNCS verification benchmark for two scenarios while (exhaustively) enumerating counterexample regions in unsafe scenarios. We also show that our approach significantly outperforms State-of-the-Art tools in closed-loop NNV.

Systems Theoretic Process Analysis of a Run Time Assured Neural Network Control System

Runtime Safety Assurance Using Reinforcement Learning

Adaptive Security Tracking Control of Constrained Unmanned Surface Vehicles

Run Time Assurance for Autonomous Spacecraft Inspection

Space Processor Computation Time Analysis for Reinforcement Learning and Run Time Assurance Control Policies

Bridging the Gap: Applying Assurance Arguments to MIL-HDBK-516C Certification of a Neural Network Control System with ASIF Run Time Assurance Architecture

Demonstrating Reinforcement Learning and Run Time Assurance for Spacecraft Inspection Using Unmanned Aerial Vehicles

Safe Spacecraft Inspection via Deep Reinforcement Learning and Discrete Control Barrier Functions

Bayesian Learning-Based Adaptive Control for Safety Critical Systems

Run Time Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems

Run Time Assured Reinforcement Learning for Six Degree-of-Freedom Spacecraft Inspection

Searching for Optimal Runtime Assurance via Reachability and Reinforcement Learning

Guaranteeing Safety for Neural Network-Based Aircraft Collision Avoidance Systems

Reachability Analysis and Safety Verification for Neural Network Control Systems

System-level Safety Guard: Safe Tracking Control through Uncertain Neural Network Dynamics Models

Reachability-Based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control

Safety Analysis of Automatic Crane Trolley Running System Based on STAMP/STPA

Case Study: Runtime Safety Verification of Neural Network Controlled System

Event-triggered fixed-time adaptive neural formation control for underactuated ASVs with connectivity constraints and prescribed performance

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Provably Safe Neural Network Controllers via Differential Dynamic Logic