Abstract:This paper proposes a novel model-free/data-driven centralized training and decentralized execution multi-agent deep reinforcement learning (MADRL) framework for distribution system voltage control with high penetration of PVs. The proposed MADRL can coordinate both the real and reactive power control of PVs with existing static var compensators and battery storage systems. Unlike the existing DRL-based voltage control methods, our proposed method does not rely on a system model during both the training and execution stages. This is achieved by developing a new interaction scheme between the surrogate modeling of the original system and the multi-agent soft actor critic (MASAC) MADRL algorithm. In particular, the sparse pseudo-Gaussian process with a few-shots of measurements is utilized to construct the surrogate model of the original environment, i.e., power flow model. This is a data-driven process and no model parameters are needed. Furthermore, the MASAC enabled MADRL allows to achieve better scalability by dividing the original system into different voltage control regions with the aid of real and reactive power sensitivities to voltage, where each region is treated as an agent. This also serves as the foundation for the centralized training and decentralized execution, thus significantly reducing the communication requirements as only local measurements are required for control. Comparative results with other alternatives on the IEEE 123-nodes and 342-nodes systems demonstrate the superiority of the proposed method.

Attention Enabled Multi-agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs

Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control with High Penetration of PVs

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

Multi-agent Graph Reinforcement Learning for Decentralized Volt-VAR Control in Power Distribution Systems

Research on Volt/Var Control of Distribution Networks Based on PPO Algorithm

Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters

A Deep Reinforcement Learning Voltage Control Method for Distribution Network

Volt-VAR Control in Active Distribution Networks Using Multi-Agent Reinforcement Learning

Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

Multi-timescale voltage control for distribution system based on multi-agent deep reinforcement learning

Deep Reinforcement Learning Based Coordinated Voltage Control in Smart Distribution Network

Two-Stage Deep Reinforcement Learning for Distribution System Voltage Regulation and Peak Demand Management

A Date-Driven Voltage Control Strategy for Distribution Network Using Deep Reinforcement Learning

A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters

Optimal Real-Time Voltage/Var Control for Distribution Network: Droop-control Based Multi-Agent Deep Reinforcement Learning

Online Multi-Agent Reinforcement Learning for Decentralized Inverter-Based Volt-VAR Control

Multi-Agent Deep Reinforcement Learning for Photovoltaics and Battery Storage Aggregators Coordinated Operation in Active Distribution Network with Incomplete Information

Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks

Three-Stage Inverter-Based Peak Shaving and Volt-VAR Control in Active Distribution Networks Using Online Safe Deep Reinforcement Learning

Reinforcement Learning Based Reactive Power Real-Time Dispatch Optimization in Distribution Networks