Abstract:With the increasing uncertainty and complexity of modern power grids, the real-time active power corrective control problem becomes intractable, bringing significant challenges to the stable operation of future power systems. To promote effective and efficient active power corrective control, a prior knowledge-embedded reinforcement learning method is proposed in this paper, to improve the performance of the deep reinforcement learning agent while maintaining the real-time control manner. The system-level feature is first established based on prior knowledge and cooperating with the equipment-level features, to provide a thorough description of the power network states. A global-local network structure is then constructed to integrate the two-level information accordingly by introducing the graph pooling method. Based on the multi-level representation of power system states, the Deep Q-learning from Demonstrations method is adopted to guide the deep reinforcement learning agent to learn from the expert policy along with the interactive improving process. Considering the infrequent corrective control actions in practice, the double-prioritized training mechanism combined with the λ-return is further developed to help the agent lay emphasis on learning from critical control experience. Simulation results demonstrate that the proposed method prevails over the conventional deep reinforcement learning methods in training efficiency and control effects, and has the potential to solve the complex active power corrective control problem in the future.

A Reinforcement Learning Approach to Power System Stabilizer

Research of Control Strategy of Power System Stabilizer Based on Reinforcement Learning

A Multi-Step Q(λ) Learning Approach to Power System Stabilizer.

An Imitation Learning and Reinforcement Learning Approach to Power System Stabilizer

Application and Development of Reinforcement Learning Theory in Power Systems

Optimal Control Method of PSS Based on Multi-Step Backtrack Q(λ) Learning

Deep Reinforcement Learning Based Approach for Active Power Security Correction Control of Power System

Design of Power System Stabilizer Based on Minimax Principle

Power System Emergency Control to Improve Short-Term Voltage Stability Using Deep Reinforcement Learning Algorithm

Reinforcement learning based CPS self-tuning control methodology for interconnected power systems

Designing of Power System Stabilizer Based on Neural-Like P Systems.

Q-learning Based Dynamic Optimal CPS Control Methodology for Interconnected Power Systems

Lyapunov-Regularized Reinforcement Learning for Power System Transient Stability

Reinforcement Learning in Adaptive Control of Power System Generation

Adaptive Stabilizing Control of Smart Transformer Based on Reinforcement Learning Optimization

Safe Reinforcement Learning for Power System Control: A Review

A prior knowledge-embedded reinforcement learning method for real-time active power corrective control in complex power systems

Deep Reinforcement Learning-Based Tie-Line Power Adjustment Method for Power System Operation State Calculation

Reactive Power Optimization Calculation Based On Multi-Step Q(Lambda) Learning Algorithm

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch