Abstract:Due to increasing penetration of volatile distributed photovoltaic (PV) resources, real-time monitoring of customers at the grid-edge has become a critical task. However, this requires solving the distribution system state estimation (DSSE) jointly for both primary and secondary levels of distribution grids, which is computationally complex and lacks scalability to large systems. To achieve near real-time solutions for DSSE, we present a novel hierarchical reinforcement learning-aided framework: at the first layer, a weighted least squares (WLS) algorithm solves the DSSE over primary medium-voltage feeders; at the second layer, deep actor-critic (A-C) modules are trained for each secondary transformer using measurement residuals to estimate the states of low-voltage circuits and capture the impact of PVs at the grid-edge. While the A-C parameter learning process takes place offline, the trained A-C modules are deployed online for fast secondary grid state estimation; this is the key factor in scalability and computational efficiency of the framework. To maintain monitoring accuracy, the two levels exchange boundary information with each other at the secondary nodes, including transformer voltages (first layer to second layer) and active/reactive total power injection (second layer to first layer). This interactive information passing strategy results in a closed-loop structure that is able to track optimal solutions at both layers in few iterations. Moreover, our model can handle the topology changes using the Jacobian matrices of the first layer. We have performed numerical experiments using real utility data and feeder models to verify the performance of the proposed framework.

Deep Reinforcement Learning-Enabled Adaptive Forecasting-Aided State Estimation in Distribution Systems with Multi-Source Multi-Rate Data

Distribution Joint State Estimation with Multiple Snapshots Based on SCADA and AMI Measurements

Time Synchronized State Estimation for Incompletely Observed Distribution Systems Using Deep Learning Considering Realistic Measurement Noise

Distributed Robust State Estimation for Hybrid AC/DC Distribution Systems using Multi-Source Data

Hierarchical deep reinforcement learning for self-adaptive economic dispatch

Distributional Deep Reinforcement Learning-Based Emegercy Frequency Control

A Hierarchical Deep Actor-Critic Learning Method for Joint Distribution System State Estimation

Deep Learning Based Forecasting-Aided State Estimation in Active Distribution Networks

A Multiarea Forecasting-Aided State Estimation Strategy for Unbalance Distribution Networks

Large Language Model-aided Edge Learning in Distribution System State Estimation

Gradient-Based Multi-Area Distribution System State Estimation

Distributed Quasi-Dynamic State Estimation Incorporating Distributed Energy Resources

Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting

A robust distribution network state estimation method based on enhanced clustering Algorithm: Accounting for multiple DG output modes and data loss

Deep Reinforcement Learning-Assisted Federated Learning for Robust Short-Term Load Forecasting in Electricity Wholesale Markets

Dynamic Energy Dispatch Strategy for Integrated Energy System Based on Improved Deep Reinforcement Learning

Learning-based State Estimation in Distribution Systems with Limited Real-Time Measurements

Deep Reinforcement Learning Framework for Resilience Enhancement of Distribution Systems under Extreme Weather Events

DistFlow Safe Reinforcement Learning Algorithm for Voltage Magnitude Regulation in Distribution Networks

Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information

A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System