Abstract:Neural Networks (NNs) have been successfully employed to represent the state evolution of complex dynamical systems. Such models, referred to as NN dynamic models (NNDMs), use iterative noisy predictions of NN to estimate a distribution of system trajectories over time. Despite their accuracy, safety analysis of NNDMs is known to be a challenging problem and remains largely unexplored. To address this issue, in this paper, we introduce a method of providing safety guarantees for NNDMs. Our approach is based on stochastic barrier functions, whose relation with safety are analogous to that of Lyapunov functions with stability. We first show a method of synthesizing stochastic barrier functions for NNDMs via a convex optimization problem, which in turn provides a lower bound on the system's safety probability. A key step in our method is the employment of the recent convex approximation results for NNs to find piece-wise linear bounds, which allow the formulation of the barrier function synthesis problem as a sum-of-squares optimization program. If the obtained safety probability is above the desired threshold, the system is certified. Otherwise, we introduce a method of generating controls for the system that robustly maximizes the safety probability in a minimally-invasive manner. We exploit the convexity property of the barrier function to formulate the optimal control synthesis problem as a linear program. Experimental results illustrate the efficacy of the method. Namely, they show that the method can scale to multi-dimensional NNDMs with multiple layers and hundreds of neurons per layer, and that the controller can significantly improve the safety probability.

Synthesizing Barrier Certificates Using Neural Networks.

Formal Synthesis of Neural Barrier Certificates for Continuous Systems Via Counterexample Guided Learning

Synthesizing Barrier Certificates of Neural Network Controlled Continuous Systems Via Approximations.

Verifying Neural Network Controlled Systems Using Neural Networks

Automated and Formal Synthesis of Neural Barrier Certificates for Dynamical Models

Synthesizing ReLU Neural Networks with Two Hidden Layers As Barrier Certificates for Hybrid Systems

Learning safe neural network controllers with barrier certificates

An Iterative Scheme of Safe Reinforcement Learning for Nonlinear Systems Via Barrier Certificate Generation

Simultaneous Synthesis and Verification of Neural Control Barrier Functions through Branch-and-Bound Verification-in-the-loop Training

Safety Certification for Stochastic Systems via Neural Barrier Functions

Safety Guarantees for Neural Network Dynamic Systems via Stochastic Barrier Functions

Polynomial Neural Barrier Certificate Synthesis of Hybrid Systems Via Counterexample Guidance

Verification-Aided Learning of Neural Network Barrier Functions with Termination Guarantees

Darboux-type barrier certificates for safety verification of nonlinear hybrid systems.

Verification of Neural Control Barrier Functions with Symbolic Derivative Bounds Propagation

A New Barrier Certificate for Safety Verification of Hybrid Systems

Safety Verification of State/time-Driven Hybrid Systems Using Barrier Certificates

Safe Reach Set Computation via Neural Barrier Certificates

Exponential-Condition-Based Barrier Certificate Generation for Safety Verification of Hybrid Systems

Learning a Formally Verified Control Barrier Function in Stochastic Environment

Accelerated Synthesis of Neural Network-based Barrier Certificates Using Collaborative Learning