Abstract:Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. For such safety-critical systems, it will certainly be a requirement that safe performance should be ensured even during the reinforcement learning period in the presence of system uncertainty. To address this issue, a Barrier Lyapunov Function-based safe reinforcement learning algorithm (BLF-SRL) is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges the Barrier Lyapunov Function item into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning when unknown bounded system uncertainty exists. More specifically, the overall system control is optimized with the optimized backstepping technique under the framework of Actor-Critic, which optimizes the virtual control in every backstepping subsystem. Wherein, the optimal virtual control is decomposed into Barrier Lyapunov Function items; and also with an adaptive item to be learned with deep neural networks, which achieves safe exploration during the learning process. Eventually, the principle of Bellman optimality is satisfied through iteratively updating the independently approximated actor and critic to solve the Hamilton-Jacobi-Bellman equation in adaptive dynamic programming. More notably, the variance of control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with motion control problems for autonomous vehicles through appropriate comparison simulations.

Safe Controller for Output Feedback Linear Systems using Model-Based Reinforcement Learning

Safe adaptive output‐feedback optimal control of a class of linear systems

Safe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

State and Input Constrained Output-Feedback Adaptive Optimal Control of Affine Nonlinear Systems

Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions

Safe Reinforcement Learning for Dynamical Systems Using Barrier Certificates

Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention

Safe Reinforcement Learning Using Robust Control Barrier Functions

Barrier Lyapunov Function-Based Safe Reinforcement Learning Algorithm for Autonomous Vehicles with System Uncertainty

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Learning for Safety-Critical Control with Control Barrier Functions

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles with Optimized Backstepping

Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions

Safe Reinforcement Learning via a Model-Free Safety Certifier

Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions

An Iterative Scheme of Safe Reinforcement Learning for Nonlinear Systems Via Barrier Certificate Generation

Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles

Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions

Safe Intermittent Reinforcement Learning for Nonlinear Systems.

Barrier Certified Safety Learning Control: When Sum-of-Square Programming Meets Reinforcement Learning