Abstract:High-performance learning-based control for the typical safety-critical autonomous vehicles invariably requires that the full-state variables are constrained within the safety region even during the learning process. To solve this technically critical and challenging problem, this work proposes an adaptive safe reinforcement learning (RL) algorithm that invokes innovative safety-related RL methods with the consideration of constraining the full-state variables within the safety region with adaptation. These are developed toward assuring the attainment of the specified requirements on the full-state variables with two notable aspects. First, thus, an appropriately optimized backstepping technique and the asymmetric barrier Lyapunov function (BLF) methodology are used to establish the safe learning framework to ensure system full-state constraints requirements. More specifically, each subsystem's control and partial derivative of the value function are decomposed with asymmetric BLF-related items and an independent learning part. Then, the independent learning part is updated to solve the Hamilton-Jacobi-Bellman equation through an adaptive learning implementation to attain the desired performance in system control. Second, with further Lyapunov-based analysis, it is demonstrated that safety performance is effectively doubly assured via a methodology of a constrained adaptation algorithm during optimization (which incorporates the projection operator and can deal with the conflict between safety and optimization). Therefore, this algorithm optimizes system control and ensures that the full set of state variables involved is always constrained within the safety region during the whole learning process. Comparison simulations and ablation studies are carried out on motion control problems for autonomous vehicles, which have verified superior performance with smaller variance and better convergence performance under uncertain circumstances. The effectiveness of the safe performance of overall system control with the proposed method accordingly has been verified.

Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control

Steady-State Error Compensation for Reinforcement Learning with Quadratic Rewards

Reinforcement Learning-Based Anti-disturbances Adaptive Control for Systems Subjected to Mismatched Disturbances and Input Uncertainties

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Performance-Guaranteed Adaptive Optimized Control of Intelligent Surface Vehicle Using Reinforcement Learning

Reinforcement learning building control approach harnessing imitation learning

Reinforcement Learning-based Active Disturbance Rejection Control for Nonlinear Systems with Disturbance

Reinforcement Learning Based Adaptive Control for Uncertain Mechanical Systems with Asymptotic Tracking

Continuous-Time Reinforcement Learning: New Design Algorithms with Theoretical Insights and Performance Guarantees

Adaptive control of a mechatronic system using constrained residual reinforcement learning

A Deep Reinforcement Learning Approach to Improve the Learning Performance in Process Control

Adaptive Optimal Consensus Control of Multiagent Systems with Unknown Dynamics and Disturbances Via Reinforcement Learning

Optimal Tracking Control of Mechatronic Servo System Using Integral Reinforcement Learning

End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks

Residual Reinforcement Learning for Robot Control

Actor-Critic Reinforcement Learning for Control With Stability Guarantee

Fault-tolerant tracking control based on reinforcement learning with application to a steer-by-wire system

Reinforcement Learning of Structured Control for Linear Systems with Unknown State Matrix

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Adaptive Safe Reinforcement Learning with Full-State Constraints and Constrained Adaptation for Autonomous Vehicles