Abstract:Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. For such safety-critical systems, it will certainly be a requirement that safe performance should be ensured even during the reinforcement learning period in the presence of system uncertainty. To address this issue, a Barrier Lyapunov Function-based safe reinforcement learning algorithm (BLF-SRL) is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges the Barrier Lyapunov Function item into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning when unknown bounded system uncertainty exists. More specifically, the overall system control is optimized with the optimized backstepping technique under the framework of Actor-Critic, which optimizes the virtual control in every backstepping subsystem. Wherein, the optimal virtual control is decomposed into Barrier Lyapunov Function items; and also with an adaptive item to be learned with deep neural networks, which achieves safe exploration during the learning process. Eventually, the principle of Bellman optimality is satisfied through iteratively updating the independently approximated actor and critic to solve the Hamilton-Jacobi-Bellman equation in adaptive dynamic programming. More notably, the variance of control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with motion control problems for autonomous vehicles through appropriate comparison simulations.

Optimal control barrier functions for RL based safe powertrain control

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems

Safe Reinforcement Learning for an Energy-Efficient Driver Assistance System

Safe Reinforcement Learning Using Robust Control Barrier Functions

Safety Filtering for Reinforcement Learning-based Adaptive Cruise Control

Safe Inverse Reinforcement Learning via Control Barrier Function

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles with Optimized Backstepping

Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings

Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions

Model-Free Safe Reinforcement Learning Through Neural Barrier Certificate

Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approach

Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles

Disturbance Observer-based Control Barrier Functions with Residual Model Learning for Safe Reinforcement Learning

Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functions

Barrier Lyapunov Function-Based Safe Reinforcement Learning Algorithm for Autonomous Vehicles with System Uncertainty

Model-Based Safe Reinforcement Learning With Time-Varying Constraints: Applications to Intelligent Vehicles

Ensuring Safety of Learning-Based Motion Planners Using Control Barrier Functions

Safe Exploration in Reinforcement Learning: Training Backup Control Barrier Functions with Zero Training Time Safety Violations

Safe Controller for Output Feedback Linear Systems using Model-Based Reinforcement Learning