Abstract:This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results in model-free DRL-based methods for power systems, but model-free methods suffer from poor sample efficiency and training time, both critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. And it is desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based-DRL framework where a deep neural network (DNN)-based dynamic surrogate model, instead of a real-world power-grid or physics-based simulation, is utilized with the policy learning framework, making the process faster and sample efficient. However, stabilizing model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We solved these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step surrogate loss. Finally, we achieved 97.5% sample efficiency and 87.7% training efficiency for an application to the IEEE 300-bus test system.

Data-Driven Preventive Control of Transient Voltage Safety

Physics-informed Evolutionary Strategy based Control for Mitigating Delayed Voltage Recovery

Distributed Deep Reinforcement Learning-based Approach for Fast Preventive Control Considering Transient Stability Constraints

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Deep Learning for Short-Term Voltage Stability Assessment of Power Systems

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

Supervised assisted deep reinforcement learning for emergency voltage control of power systems

Power System Emergency Control to Improve Short-Term Voltage Stability Using Deep Reinforcement Learning Algorithm

A Safe Pre-Decision Making Approach for Power System Voltage Stability Enhancement Via Safe Deep Reinforcement Learning

Intelligent Short-Term Voltage Stability Assessment via Spatial Attention Rectified RNN Learning

Research on Power System Transient Stability Assessment Based on Deep Learning of Big Data Technique

Intelligent Time-Adaptive Transient Stability Assessment System

A Hierarchical Self-Adaptive Data-Analytics Method for Real-Time Power System Short-Term Voltage Stability Assessment

Safe Reinforcement Learning for Emergency LoadShedding of Power Systems

Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning

Deep-Reinforcement-Learning-Based Two-Timescale Voltage Control for Distribution Systems

An Intelligent Transient Stability Assessment Framework With Continual Learning Ability

Transient Voltage Stability Assessment Method for the UHVDC Power Grid with High Proportion of New Energy Based on Artificial Intelligence

A New Transient Voltage Stability Prediction Model Using Big Data Analysis

Reactive Power Optimization for Transient Voltage Stability in Energy Internet Via Deep Reinforcement Learning Approach

A novel data-driven approach for transient stability prediction of power systems considering the operational variability