Abstract:Improving system-level resiliency of networked microgrids is an important aspect with increased population of inverter-based resources (IBRs). This paper (1) presents resilient control design in presence of adversarial cyber-events, and proposes a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issues regarding data sharing in multi-party-owned networked grids, and (2) transfers learned controls from simulation to hardware-in-the-loop test-bed, thereby bridging the gap between simulation and real world. With these multi-prong objectives, first, we formulate a reinforcement learning (RL) training setup generating episodic trajectories with adversaries (attack signal) injected at the primary controllers of the grid forming (GFM) inverters where RL agents (or controllers) are being trained to mitigate the injected attacks. For networked microgrids, the horizontal Fed-RL method involving distinct independent environments is not appropriate, leading us to develop vertical variant Federated Soft Actor-Critic (FedSAC) algorithm to grasp the interconnected dynamics of networked microgrid. Next, utilizing OpenAI Gym interface, we built a custom simulation set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark test systems comprising 3 interconnected microgrids. Finally, the learned policies in simulation world are transferred to the real-time hardware-in-the-loop test-bed set-up developed using high-fidelity Hypersim platform. Experiments show that the simulator-trained RL controllers produce convincing results with the real-time test-bed set-up, validating the minimization of sim-to-real gap.

Increasing the Flexibility of Hydropower with Reinforcement Learning on a Digital Twin Platform

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates

Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control

Application of Deep Reinforcement Learning Control of an Inverted Hydraulic Pendulum

Adaptive Power System Emergency Control using Deep Reinforcement Learning

Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks

Efficient Reservoir Management through Deep Reinforcement Learning

Deep Reinforcement Learning for Long-Term Voltage Stability Control

Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach with Safe Gradient Flow

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Safe Reinforcement Learning for Power System Control: A Review

Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation

Research on low-energy consumption automatic real-time regulation of cascade gates and pumps in open-canal based on reinforcement learning

Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning

Rolling horizon wind-thermal unit commitment optimization based on deep reinforcement learning

Safe Reinforcement Learning for Emergency LoadShedding of Power Systems

Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Flood mitigation in coastal urban catchments using real-time stormwater infrastructure control and reinforcement learning

Human–Machine Collaborative Reinforcement Learning for Power Line Flow Regulation

Cooling Water System Control of a Commercial Complex Based on Reinforcement Learning