Abstract:High-performance learning-based control for the typical safety-critical autonomous vehicles invariably requires that the full-state variables are constrained within the safety region even during the learning process. To solve this technically critical and challenging problem, this work proposes an adaptive safe reinforcement learning (RL) algorithm that invokes innovative safety-related RL methods with the consideration of constraining the full-state variables within the safety region with adaptation. These are developed toward assuring the attainment of the specified requirements on the full-state variables with two notable aspects. First, thus, an appropriately optimized backstepping technique and the asymmetric barrier Lyapunov function (BLF) methodology are used to establish the safe learning framework to ensure system full-state constraints requirements. More specifically, each subsystem's control and partial derivative of the value function are decomposed with asymmetric BLF-related items and an independent learning part. Then, the independent learning part is updated to solve the Hamilton-Jacobi-Bellman equation through an adaptive learning implementation to attain the desired performance in system control. Second, with further Lyapunov-based analysis, it is demonstrated that safety performance is effectively doubly assured via a methodology of a constrained adaptation algorithm during optimization (which incorporates the projection operator and can deal with the conflict between safety and optimization). Therefore, this algorithm optimizes system control and ensures that the full set of state variables involved is always constrained within the safety region during the whole learning process. Comparison simulations and ablation studies are carried out on motion control problems for autonomous vehicles, which have verified superior performance with smaller variance and better convergence performance under uncertain circumstances. The effectiveness of the safe performance of overall system control with the proposed method accordingly has been verified.

Barrier Lyapunov Function-Based Safe Reinforcement Learning for Autonomous Vehicles with Optimized Backstepping

Barrier Lyapunov Function-Based Safe Reinforcement Learning Algorithm for Autonomous Vehicles with System Uncertainty

Adaptive Safe Reinforcement Learning with Full-State Constraints and Constrained Adaptation for Autonomous Vehicles

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approach

Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles

Model-Based Safe Reinforcement Learning With Time-Varying Constraints: Applications to Intelligent Vehicles

Ensuring Safety of Learning-Based Motion Planners Using Control Barrier Functions

Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions

Safe Exploration in Reinforcement Learning: Training Backup Control Barrier Functions with Zero Training Time Safety Violations

Optimal control barrier functions for RL based safe powertrain control

Safe Controller for Output Feedback Linear Systems using Model-Based Reinforcement Learning

Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions

Model-Free Safe Reinforcement Learning Through Neural Barrier Certificate

Enhancing System-Level Safety in Mixed-Autonomy Platoon via Safe Reinforcement Learning

Barrier Lyapunov function-based adaptive optimized control for full-state and input-constrained dynamic positioning of marine vessels with simulation and model-scale tests

Safe Reinforcement Learning for Model-Reference Trajectory Tracking of Uncertain Autonomous Vehicles with Model-Based Acceleration

Robust Reinforcement Learning with UUB Guarantee for Safe Motion Control of Autonomous Robots

Safe Reinforcement Learning for Dynamical Systems Using Barrier Certificates