Abstract:Safe Reinforcement learning (RL) has been widely investigated to conduct power systems frequency control under high renewable energy resources penetration. Nevertheless, existing safe RL-based frequency control methods still face two fundamental challenges to achieving safety guarantees: (1) operating in non-stationary environments without the prior knowledge of the system parameters and (2) simultaneously satisfying high-dimensional and time-varying safety constraints in multi-area cases. To this end, this paper proposes a prior-free reinforcement learning-based frequency control method with guaranteed safety for multi-area power systems named AdapSafe2. To tackle Challenge (1), a meta-based environmental learning algorithm is developed to automatically capture and rapidly adapts to non-stationary system parameters without relying on a predefined nominal model. Furthermore, a meta-RL framework is established to achieve a self-adaptive frequency control strategy without prior knowledge. Moreover, for Challenge (2), a novel safety-critic network and a safe-certified compensator based on the control barrier function are designed to identify time-varying safety constraints. Leveraging risk assessments from the safety-critic network, the compensator performs dynamic safety compensations only for areas with risk, thereby enhancing the efficiency of solving under high-dimensional safety constraints. Numerical simulations conducted under 2-Area and 3-Area wind-aggregated low-inertia power systems demonstrate that the proposed AdapSafe2 can outperform the state-of-the-art approaches while effectively satisfying the dynamic safety constraints.

Dynamic Nonlinear Droop-Based Fast Frequency Regulation for Power Systems with Flexible Resources Using Meta-Reinforcement Learning Approach

AdapSafe: Adaptive and Safe-Certified Deep Reinforcement Learning-Based Frequency Control for Carbon-Neutral Power Systems.

AdapSafe2: Prior-Free Safe-Certified Reinforcement Learning for Multi-Area Frequency Control

Fast Frequency Regulation Utilizing Non-Aggregate Thermostatically Controlled Loads Based on Edge Intelligent Terminals

Exploiting Demand-Side Heterogeneous Flexible Resources in Risk Management of Power System Frequency

Reinforcement Learning-Based Inertia and Droop Control for Wind Farm Frequency Regulation

A Deep Reinforcement Learning Based Framework for Power System Load Frequency Control

Modeling and Control of Flexible Loads for Frequency Regulation Services Considering Compensation of Communication Latency and Detection Error

Computationally Efficient Safe Reinforcement Learning for Power Systems

A Reinforcement Learning Based Coordinated but Differentiated Load Frequency Control Method With Heterogeneous Frequency Regulation Resources

Adaptive Frequency Response from Electrified Railway

Model-Free Load Frequency Control of Nonlinear Power Systems Based on Deep Reinforcement Learning

Learning and Fast Adaptation for Grid Emergency Control via Deep Meta Reinforcement Learning

A Deep Reinforcement Learning-Embedded Two-Stage Framework for Performance-Sensitive Load Frequency Control

Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning

Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guarantees

Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach with Safe Gradient Flow

Frequency Regulation System: A Deep Learning Identification, Type-3 Fuzzy Control and LMI Stability Analysis

Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control

Rule-Based Adaptive Frequency Regulation With Real Stochastic Model Intermittency in a Restructured Power System