Abstract:This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

Application of intelligent system based on deep reinforcement learning in electrical engineering automation control

Emotional Deep Learning Programming Controller for Automatic Voltage Control of Power Systems

Deep Reinforcement Learning Based Approach for Active Power Security Correction Control of Power System

Artificial emotionnal Q-learning for automatic generation control of interconnected power grids

Deep Reinforcement Learning for Power System Applications: An Overview

Application and Development of Reinforcement Learning Theory in Power Systems

A prior knowledge-embedded reinforcement learning method for real-time active power corrective control in complex power systems

Adaptive Power System Emergency Control using Deep Reinforcement Learning

Supervised assisted deep reinforcement learning for emergency voltage control of power systems

Artificial Emotional Reinforcement Learning for Automatic Generation Control of Large-Scale Interconnected Power Grids

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations

An Application of Reinforcement Learning to Voltage Control in Power System

Reinforcement Learning Optimized Intelligent Electricity Dispatching System

A Human-Machine Reinforcement Learning Method for Cooperative Energy Management.

A Deep Reinforcement Learning Framework for Automatic Operation Control of Power System Considering Extreme Weather Events

Data-driven active corrective control in power systems: an interpretable deep reinforcement learning approach

On Training Effective Reinforcement Learning Agents for Real-time Power Grid Operation and Control

Emergency Control Methods for Power Systems Based on Improved Deep Reinforcement Learning

Artificial emotional deep Q learning for real-time smart voltage control of cyber-physical social power systems

Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning