Abstract:Active distribution networks are being challenged by frequent and rapid voltage violations due to renewable energy integration. Conventional model-based voltage control methods rely on accurate parameters of the distribution networks, which are difficult to achieve in practice. This paper proposes a novel physical-model-free two-timescale voltage control framework for active distribution systems. To achieve fast control of PV inverters, the whole network is first partitioned into several sub-networks using voltage-reactive power sensitivity. Then, the scheduling of PV inverters in the multiple sub-networks is formulated as Markov games and solved by a multi-agent soft actor-critic (MASAC) algorithm, where each sub-network is modeled as an intelligent agent. All agents are trained in a centralized manner to learn a coordinated strategy while being executed based on only local information for fast response. For the slower time-scale control, OLTCs and switched capacitors are coordinated by a single agent-based SAC algorithm using the global information with considering control behaviors of the inverters. Particularly, the two-level agents are trained concurrently with information exchange according to the reward signal calculated from the data-driven surrogate model. Comparative tests with different benchmark methods on IEEE 33- and 123-bus systems and 342-node low voltage distribution system demonstrate that the proposed method can effectively mitigate the fast voltage violations and achieve systematical coordination of different voltage regulation assets without the knowledge of accurate system model.

Online Volt-Var Control Based on Reinforcement Learning in Active Distribution Systems Considering PV Inverters

Research on Volt/Var Control of Distribution Networks Based on PPO Algorithm

Online Voltage Control Strategy: Multi-Mode Based Data-Driven Approach for Active Distribution Networks

Data-Driven Robust Voltage/VAR Control Using PV Inverters in Active Distribution Networks

Volt-VAR Control in Active Distribution Networks Using Multi-Agent Reinforcement Learning

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control

Multi-Agent Deep Reinforcement Learning for Realistic Distribution System Voltage Control Using PV Inverters

Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

Voltage Control of Distribution Grid with District Cooling Systems Based on Scenario-Classified Reinforcement Learning

Fair Reinforcement Learning Algorithm for PV Active Control in LV Distribution Networks

Two-Stage Deep Reinforcement Learning for Distribution System Voltage Regulation and Peak Demand Management

Three-Stage Inverter-Based Peak Shaving and Volt-VAR Control in Active Distribution Networks Using Online Safe Deep Reinforcement Learning

Reinforcement Learning Based Robust Volt/Var Control in Active Distribution Networks With Imprecisely Known Delay

Reinforcement Learning for Volt-Var Control: A Novel Two-stage Progressive Training Strategy

A Deep Reinforcement Learning Voltage Control Method for Distribution Network

Two-Stage Deep Reinforcement Learning for Inverter-Based Volt-VAR Control in Active Distribution Networks

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

Optimal Real-Time Voltage/Var Control for Distribution Network: Droop-control Based Multi-Agent Deep Reinforcement Learning

Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control

Robust Offline Deep Reinforcement Learning for Volt-Var Control in Active Distribution Networks