Abstract:Reinforcement Learning (RL) and continuous nonlinear control have been successfully deployed in multiple domains of complicated sequential decision-making tasks. However, given the exploration nature of the learning process and the presence of model uncertainty, it is challenging to apply them to safety-critical control tasks due to the lack of safety guarantee. On the other hand, while combining control-theoretical approaches with learning algorithms has shown promise in safe RL applications, the sample efficiency of safe data collection process for control is not well addressed. In this paper, we propose a \emph{provably} sample efficient episodic safe learning framework for online control tasks that leverages safe exploration and exploitation in an unknown, nonlinear dynamical system. In particular, the framework 1) extends control barrier functions (CBFs) in a stochastic setting to achieve provable high-probability safety under uncertainty during model learning and 2) integrates an optimism-based exploration strategy to efficiently guide the safe exploration process with learned dynamics for \emph{near optimal} control performance. We provide formal analysis on the episodic regret bound against the optimal controller and probabilistic safety with theoretical guarantees. Simulation results are provided to demonstrate the effectiveness and efficiency of the proposed algorithm.

Control Barrier Function Augmentation in Sampling-based Control Algorithm for Sample Efficiency

Path Integral Methods with Stochastic Control Barrier Functions

Whole-body Dynamic Collision Avoidance with Time-varying Control Barrier Functions

Safe Tracking Control for Free-Flying Space Robots Via Control Barrier Functions

Resilient Estimator-based Control Barrier Functions for Dynamical Systems with Disturbances and Noise

Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions

Safety-Critical Planning and Control for Dynamic Obstacle Avoidance Using Control Barrier Functions

Uniform Finite Time Safe Path Tracking Control for Obstacle Avoidance of Autonomous Vehicle Via Barrier Function Approach

Safe Importance Sampling in Model Predictive Path Integral Control

Guaranteed-Safe MPPI Through Composite Control Barrier Functions for Efficient Sampling in Multi-Constrained Robotic Systems

Safety of Sampled-Data Systems with Control Barrier Functions via Approximate Discrete Time Models

Optimal control of connected automated vehicles with event/self-triggered control barrier functions

Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Function

Adaptive Safety-Critical Control for a Class of Nonlinear Systems with Parametric Uncertainties: A Control Barrier Function Approach

Predictive Control Barrier Functions: Enhanced Safety Mechanisms for Learning-Based Control

On the Optimality, Stability, and Feasibility of Control Barrier Functions: An Adaptive Learning-Based Approach

Time-Varying Soft-Maximum Barrier Functions for Safety in Unmapped and Dynamic Environments

Safety-Critical Control Synthesis for Unknown Sampled-Data Systems via Control Barrier Functions

Control Barrier Function Meets Interval Analysis: Safety-Critical Control with Measurement and Actuation Uncertainties

Probabilistically safe controllers based on control barrier functions and scenario model predictive control

Quantifying Safety of Learning-based Self-Driving Control Using Almost-Barrier Functions