Abstract:Ensuring safety of nonlinear systems under model uncertainty and external disturbances is crucial, especially for real-world control tasks. Predictive methods such as robust model predictive control (RMPC) require solving nonconvex optimization problems online, which leads to high computational burden and poor scalability. Reinforcement learning (RL) works well with complex systems, but pays the price of losing rigorous safety guarantee. This paper presents a theoretical framework that bridges the advantages of both RMPC and RL to synthesize safety filters for nonlinear systems with state- and action-dependent uncertainty. We decompose the robust invariant set (RIS) into two parts: a target set that aligns with terminal region design of RMPC, and a reach-avoid set that accounts for the rest of RIS. We propose a policy iteration approach for robust reach-avoid problems and establish its monotone convergence. This method sets the stage for an adversarial actor-critic deep RL algorithm, which simultaneously synthesizes a reach-avoid policy network, a disturbance policy network, and a reach-avoid value network. The learned reach-avoid policy network is utilized to generate nominal trajectories for online verification, which filters potentially unsafe actions that may drive the system into unsafe regions when worst-case disturbances are applied. We formulate a second-order cone programming (SOCP) approach for online verification using system level synthesis, which optimizes for the worst-case reach-avoid value of any possible trajectories. The proposed safety filter requires much lower computational complexity than RMPC and still enjoys persistent robust safety guarantee. The effectiveness of our method is illustrated through a numerical example.

Predictive safety filter using system level synthesis

Stability Mechanisms for Predictive Safety Filters

Adaptive Model Predictive Safety Certification for Learning-based Control -- Extended Version

Learning Predictive Safety Filter via Decomposition of Robust Invariant Set

A Robust, Efficient Predictive Safety Filter

Predictive Control Barrier Functions: Enhanced Safety Mechanisms for Learning-Based Control

Advanced safety filter based on SOS Control Barrier and Lyapunov Functions

Data-Driven Safety Filter: An Input-Output Perspective

Safe Interactive Model-Based Learning

Q-learning-based Model-free Safety Filter

Probabilistically safe controllers based on control barrier functions and scenario model predictive control

Multi-Step Model Predictive Safety Filters: Reducing Chattering by Increasing the Prediction Horizon

Safe Learning-Based Model Predictive Control Using The Compatible Models Approach

A Modular Safety Filter for Safety-Certified Cyber-Physical Systems

Predictive stability filters for nonlinear dynamical systems affected by disturbances

Safety Filters for Black-Box Dynamical Systems by Learning Discriminating Hyperplanes

Safety Filter Design for Neural Network Systems via Convex Optimization

Safe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties

Safe Importance Sampling in Model Predictive Path Integral Control

Distributed Model Predictive Safety Certification for Learning-based Control

Formal Control Synthesis Via Safe Reinforcement Learning under Real-Time Specifications