Abstract:The main goal of active voltage control is to keep the voltage of each bus in the grid within a safe range. With the increasing penetration of renewable and distributed energy sources in the grid, growing complexity, increasing uncertainty, and aggravating volatility bring great challenges to voltage control in modern power systems. Traditional algorithms can hardly guarantee real-time safe control to cope with these challenges. In recent years, substantial attention has been paid to the application of multi-agent reinforcement learning algorithms (MARL) to coordinate the control units in each area of the grid in real time for active voltage control in complex scenarios. However, these MARL algorithms do not explicitly guarantee that the power system satisfies the security constraints. There is a little in-depth study on safe multi-agent policy learning in multi-agent-based voltage control, especially the direct correction of unsafe actions. In this paper, we formalize the active voltage control problem as a Constrained Markov Game and approach it with a centralized data-driven safety layer that requires global observations and maps unsafe actions to safe actions. In order to make the policy network rely on local observations for decentralized execution, we introduce two novel components into the policy network: action correction penalty loss and action correction sub-networks. Notably, our approaches are easily extendable to other MARL algorithms for continuous actions. In the experiments, we evaluate our methods in the power distribution network simulation environment and demonstrate the capability of the safety layer to correct unsafe actions and the effectiveness of the safety layer to improve the performance of the policy itself.

A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning

Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning

Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control with High Penetration of PVs

A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control

Multi-agent Graph Reinforcement Learning for Decentralized Volt-VAR Control in Power Distribution Systems

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

DDPG-Based Multi-Agent Framework for SVC Tuning in Urban Power Grid with Renewable Energy Resources

Multi-timescale voltage control for distribution system based on multi-agent deep reinforcement learning

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

Multi-Agent Reinforcement Learning with Safety Layer for Active Voltage Control

Online Voltage Control Strategy: Multi-Mode Based Data-Driven Approach for Active Distribution Networks

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

A Date-Driven Voltage Control Strategy for Distribution Network Using Deep Reinforcement Learning

A Deep Reinforcement Learning Voltage Control Method for Distribution Network

Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework

Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning

Multi-Agent Graph-Attention Deep Reinforcement Learning for Post-Contingency Grid Emergency Voltage Control

Graph Multi-Agent Reinforcement Learning for Inverter-Based Active Voltage Control

Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters