Abstract:The main goal of active voltage control is to keep the voltage of each bus in the grid within a safe range. With the increasing penetration of renewable and distributed energy sources in the grid, growing complexity, increasing uncertainty, and aggravating volatility bring great challenges to voltage control in modern power systems. Traditional algorithms can hardly guarantee real-time safe control to cope with these challenges. In recent years, substantial attention has been paid to the application of multi-agent reinforcement learning algorithms (MARL) to coordinate the control units in each area of the grid in real time for active voltage control in complex scenarios. However, these MARL algorithms do not explicitly guarantee that the power system satisfies the security constraints. There is a little in-depth study on safe multi-agent policy learning in multi-agent-based voltage control, especially the direct correction of unsafe actions. In this paper, we formalize the active voltage control problem as a Constrained Markov Game and approach it with a centralized data-driven safety layer that requires global observations and maps unsafe actions to safe actions. In order to make the policy network rely on local observations for decentralized execution, we introduce two novel components into the policy network: action correction penalty loss and action correction sub-networks. Notably, our approaches are easily extendable to other MARL algorithms for continuous actions. In the experiments, we evaluate our methods in the power distribution network simulation environment and demonstrate the capability of the safety layer to correct unsafe actions and the effectiveness of the safety layer to improve the performance of the policy itself.

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

Deep Reinforcement Learning Based Coordinated Voltage Control in Smart Distribution Network

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Volt-VAR Control in Active Distribution Networks Using Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks

Online Voltage Control Strategy: Multi-Mode Based Data-Driven Approach for Active Distribution Networks

Graph Multi-Agent Reinforcement Learning for Inverter-Based Active Voltage Control

Multi-timescale voltage control for distribution system based on multi-agent deep reinforcement learning

Multi-Agent Reinforcement Learning with Safety Layer for Active Voltage Control

Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control

Robust Regional Coordination of Inverter-Based Volt/Var Control Via Multi-Agent Deep Reinforcement Learning

A Deep Reinforcement Learning Voltage Control Method for Distribution Network

Multi-agent Edge-Learning Based Real-Time Voltage Control for Active Distribution Network with Prediction Uncertainties Tolerance

Coordinated Electric Vehicle Active and Reactive Power Control for Active Distribution Networks

A MATD3 -based Voltage Control Strategy for Distribution Networks Considering Active and Reactive Power Adjustment Costs

Coordinated voltage regulation of high renewable-penetrated distribution networks: An evolutionary curriculum-based deep reinforcement learning approach

Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters

A Date-Driven Voltage Control Strategy for Distribution Network Using Deep Reinforcement Learning

Secondary Voltage Collaborative Control of Distributed Energy System via Multi-Agent Reinforcement Learning

Two-Stage Multi-Mode Voltage Control for Distribution Networks: A Deep Reinforcement Learning Approach Based on Multiple Intelligences