Abstract:High-performance learning-based control for the typical safety-critical autonomous vehicles invariably requires that the full-state variables are constrained within the safety region even during the learning process. To solve this technically critical and challenging problem, this work proposes an adaptive safe reinforcement learning (RL) algorithm that invokes innovative safety-related RL methods with the consideration of constraining the full-state variables within the safety region with adaptation. These are developed toward assuring the attainment of the specified requirements on the full-state variables with two notable aspects. First, thus, an appropriately optimized backstepping technique and the asymmetric barrier Lyapunov function (BLF) methodology are used to establish the safe learning framework to ensure system full-state constraints requirements. More specifically, each subsystem's control and partial derivative of the value function are decomposed with asymmetric BLF-related items and an independent learning part. Then, the independent learning part is updated to solve the Hamilton-Jacobi-Bellman equation through an adaptive learning implementation to attain the desired performance in system control. Second, with further Lyapunov-based analysis, it is demonstrated that safety performance is effectively doubly assured via a methodology of a constrained adaptation algorithm during optimization (which incorporates the projection operator and can deal with the conflict between safety and optimization). Therefore, this algorithm optimizes system control and ensures that the full set of state variables involved is always constrained within the safety region during the whole learning process. Comparison simulations and ablation studies are carried out on motion control problems for autonomous vehicles, which have verified superior performance with smaller variance and better convergence performance under uncertain circumstances. The effectiveness of the safe performance of overall system control with the proposed method accordingly has been verified.

AdapSafe2: Prior-Free Safe-Certified Reinforcement Learning for Multi-Area Frequency Control

AdapSafe: Adaptive and Safe-Certified Deep Reinforcement Learning-Based Frequency Control for Carbon-Neutral Power Systems.

Highly transferable adversarial attack against deep‐reinforcement‐learning‐based frequency control

Computationally Efficient Safe Reinforcement Learning for Power Systems

Safe Reinforcement Learning for Power System Control: A Review

Safe Exploration in Wireless Security: A Safe Reinforcement Learning Algorithm With Hierarchical Structure

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

Adaptive Safe Reinforcement Learning with Full-State Constraints and Constrained Adaptation for Autonomous Vehicles

Safe Reinforcement Learning for Active Distribution Networks Reconfiguration Considering Uncertainty

Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates

Rethinking Safe Policy Learning for Complex Constraints Satisfaction: A Glimpse in Real-Time Security Constrained Economic Dispatch Integrating Energy Storage Units

A Review of Safe Reinforcement Learning Methods for Modern Power Systems

A Safe Reinforcement Learning Algorithm for Supervisory Control of Power Plants

Model-Based Safe Reinforcement Learning With Time-Varying Constraints: Applications to Intelligent Vehicles

A Safe Pre-Decision Making Approach for Power System Voltage Stability Enhancement Via Safe Deep Reinforcement Learning

Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control

Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guarantees

A Load Frequency Control Strategy Based on Double Deep Q-network and Upper Confidence Bound Algorithm of Multi-Area Interconnected Power Systems

A Safe Policy Learning-Based Method for Decentralized and Economic Frequency Control in Isolated Networked-Microgrid Systems

Safe Reinforcement Learning for Emergency LoadShedding of Power Systems

Contingency-constrained economic dispatch with safe reinforcement learning