Abstract:This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

Deep Forest Reinforcement Learning for Preventive Strategy Considering Automatic Generation Control in Large-Scale Interconnected Power Systems

Exploring the Vulnerability of Deep Reinforcement Learning-based Emergency Control for Low Carbon Power Systems

Adaptive Power System Emergency Control using Deep Reinforcement Learning

Deep Reinforcement Learning Based Approach for Active Power Security Correction Control of Power System

Distributed Hierarchical Deep Reinforcement Learning for Large-Scale Grid Emergency Control

Automatic Generation Control Strategy Based on Deep Forest.

Distributed Deep Reinforcement Learning-based Approach for Fast Preventive Control Considering Transient Stability Constraints

Distributional Deep Reinforcement Learning-Based Emegercy Frequency Control

Supervised assisted deep reinforcement learning for emergency voltage control of power systems

A Deep Reinforcement Learning Framework for Automatic Operation Control of Power System Considering Extreme Weather Events

Transient Stability Emergency Control Strategy of Power System Based on Deep Reinforcement Learning

AdapSafe: Adaptive and Safe-Certified Deep Reinforcement Learning-Based Frequency Control for Carbon-Neutral Power Systems.

Emergency fault affected wide-area automatic generation control via large-scale deep reinforcement learning

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Learning and Fast Adaptation for Grid Emergency Control via Deep Meta Reinforcement Learning

Artificial Emotional Reinforcement Learning for Automatic Generation Control of Large-Scale Interconnected Power Grids

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Double DQN-based Power System Transient Stability Emergency Control with Protection Coordinations

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Hierarchical deep reinforcement learning for self-adaptive economic dispatch

Deep Reinforcement Learning Framework for Resilience Enhancement of Distribution Systems under Extreme Weather Events