Abstract:The main goal of active voltage control is to keep the voltage of each bus in the grid within a safe range. With the increasing penetration of renewable and distributed energy sources in the grid, growing complexity, increasing uncertainty, and aggravating volatility bring great challenges to voltage control in modern power systems. Traditional algorithms can hardly guarantee real-time safe control to cope with these challenges. In recent years, substantial attention has been paid to the application of multi-agent reinforcement learning algorithms (MARL) to coordinate the control units in each area of the grid in real time for active voltage control in complex scenarios. However, these MARL algorithms do not explicitly guarantee that the power system satisfies the security constraints. There is a little in-depth study on safe multi-agent policy learning in multi-agent-based voltage control, especially the direct correction of unsafe actions. In this paper, we formalize the active voltage control problem as a Constrained Markov Game and approach it with a centralized data-driven safety layer that requires global observations and maps unsafe actions to safe actions. In order to make the policy network rely on local observations for decentralized execution, we introduce two novel components into the policy network: action correction penalty loss and action correction sub-networks. Notably, our approaches are easily extendable to other MARL algorithms for continuous actions. In the experiments, we evaluate our methods in the power distribution network simulation environment and demonstrate the capability of the safety layer to correct unsafe actions and the effectiveness of the safety layer to improve the performance of the policy itself.

Multi-Agent Q-Value Mixing Network with Covariance Matrix Adaptation Strategy for the Voltage Regulation Problem

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

Robust Regional Coordination of Inverter-Based Volt/Var Control Via Multi-Agent Deep Reinforcement Learning

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Population-based Multi-agent Evaluation for Large-scale Voltage Control.

Deep Reinforcement Learning Based Coordinated Voltage Control in Smart Distribution Network

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Multi-Agent Reinforcement Learning with Safety Layer for Active Voltage Control

A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters

Multi Agent Safe Graph Reinforcement Learning for PV Inverter s Based Real-Time De centralized Volt/Var Control in Zoned Distribution Networks

Coordinated Control of Multiple Converters in Model-Free AC/DC Distribution Networks Based on Reinforcement Learning

Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks

Multiagent-Based Reinforcement Learning for Optimal Reactive Power Dispatch.

Learning Multi-Agent Cooperation via Considering Actions of Teammates

Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

QMIX-based Multi-Agent Reinforcement Learning for Electric Vehicle-Facilitated Peak Shaving

Multi-regional Reactive Power Optimization Based on Correlated Equilibrium Q-learning Collaborative Algorithm

Multi-Agent Q($$ \lambda $$) Learning for Optimal Operation Management of Energy Internet

Online Voltage Control Strategy: Multi-Mode Based Data-Driven Approach for Active Distribution Networks

Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems