Abstract:The high proportion of renewable energy integration and the dynamic changes in grid topology necessitate the enhancement of voltage/var control (VVC) to manage voltage fluctuations more rapidly. Traditional model-based control algorithms are becoming increasingly incompetent for VVC due to their high model dependence and slow online computation speed. To alleviate these issues, this paper introduces a graph attention network (GAT) based deep reinforcement learning for VVC of topologically variable power systems. Firstly, combining the physical information of the actual power grid, a physics-informed GAT is proposed and embedded into the proximal policy optimization (PPO) algorithm. It can capture topological and spatial correlations among the node features to tackle topology changes. To address the slow training, the improved ReliefF algorithm identifies critical state variables, significantly reducing the dimensionality of state space. Then, the training samples retained in the experience buffer are designed to mitigate the sparse reward issue. Finally, validation on the modified IEEE 39-bus system and an actual power grid demonstrate superior performance compared with state-of-the-art methods, including PPO algorithm and twin delayed deep deterministic policy gradient (TD3) algorithm. The proposed algorithm exhibits enhanced convergence during training, faster solution speed, and improved VVC performance, even in scenarios involving grid topology changes and increased renewable energy integration. Meanwhile, in the adopted cases, the network loss is reduced by 6.9%, 10.8%, and 7.7%, respectively, demonstrating favorable economic outcomes.

An Application of Reinforcement Learning to Voltage Control in Power System

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

A Two-Level System-Wide Automatic Voltage Control System

Power Grid Reactive Voltage Control Method Based on Improved Reinforcement Learning

Power System Emergency Control to Improve Short-Term Voltage Stability Using Deep Reinforcement Learning Algorithm

Deep-Reinforcement-Learning-Based Two-Timescale Voltage Control for Distribution Systems

Deep Reinforcement Learning Based Voltage Control Revisited

Application and Development of Reinforcement Learning Theory in Power Systems

Application of Reinforcement Learning Method in a Hydro-thermal Hybrid Automatic Generation Control System

Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control

Simplified Deep Reinforcement Learning Based Volt-var Control of Topologically Variable Power System

Graph Attention Network Based Deep Reinforcement Learning for Voltage/var Control of Topologically Variable Power System

Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning

APPLICATION OF REINFORCEMENT LEARNING TO STATCOM CONTROLLER

Reinforcement Learning-based District Cooling System Control for Voltage Regulation in Distribution Networks.

Deep Reinforcement Leaming for Short-term Voltage Control by Dynamic Load Shedding in China Southem Power Grid

Online Volt-Var Control Based on Reinforcement Learning in Active Distribution Systems Considering PV Inverters

Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning

Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach with Safe Gradient Flow

Voltage Control of Distribution Grid with District Cooling Systems Based on Scenario-Classified Reinforcement Learning

Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations