Abstract:This paper proposes a novel model-free/data-driven centralized training and decentralized execution multi-agent deep reinforcement learning (MADRL) framework for distribution system voltage control with high penetration of PVs. The proposed MADRL can coordinate both the real and reactive power control of PVs with existing static var compensators and battery storage systems. Unlike the existing DRL-based voltage control methods, our proposed method does not rely on a system model during both the training and execution stages. This is achieved by developing a new interaction scheme between the surrogate modeling of the original system and the multi-agent soft actor critic (MASAC) MADRL algorithm. In particular, the sparse pseudo-Gaussian process with a few-shots of measurements is utilized to construct the surrogate model of the original environment, i.e., power flow model. This is a data-driven process and no model parameters are needed. Furthermore, the MASAC enabled MADRL allows to achieve better scalability by dividing the original system into different voltage control regions with the aid of real and reactive power sensitivities to voltage, where each region is treated as an agent. This also serves as the foundation for the centralized training and decentralized execution, thus significantly reducing the communication requirements as only local measurements are required for control. Comparative results with other alternatives on the IEEE 123-nodes and 342-nodes systems demonstrate the superiority of the proposed method.

Reinforcement Learning Based Reactive Power Real-Time Dispatch Optimization in Distribution Networks

Research on Volt/Var Control of Distribution Networks Based on PPO Algorithm

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Multi-Agent Reinforcement Learning-based Distributed Economic Dispatch Considering Network attacks and Uncertain Costs

DDPG-based Reactive Power Optimization Strategy for Active Distribution Network

A Deep Reinforcement Learning Voltage Control Method for Distribution Network

Optimal Real-Time Voltage/Var Control for Distribution Network: Droop-control Based Multi-Agent Deep Reinforcement Learning

Multiagent-Based Reinforcement Learning for Optimal Reactive Power Dispatch.

Reactive Power Optimization of Distribution Network Based on Deep Reinforcement Learning and Multi Agent System

Deep Reinforcement Learning Based Optimal Scheduling of Active Distribution System Considering Distributed Generation, Energy Storage and Flexible Load

Deep Reinforcement Learning Based Coordinated Voltage Control in Smart Distribution Network

Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm

Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

A Date-Driven Voltage Control Strategy for Distribution Network Using Deep Reinforcement Learning

Volt-VAR Control in Active Distribution Networks Using Multi-Agent Reinforcement Learning

Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control with High Penetration of PVs

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

Real-time Operation Optimization in Active Distribution Networks Based on Multi-Agent Deep Reinforcement Learning

Robust Regional Coordination of Inverter-Based Volt/Var Control Via Multi-Agent Deep Reinforcement Learning

Reinforcement Learning Based Robust Volt/Var Control in Active Distribution Networks With Imprecisely Known Delay

Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters