Abstract:Improving system-level resiliency of networked microgrids is an important aspect with increased population of inverter-based resources (IBRs). This paper (1) presents resilient control design in presence of adversarial cyber-events, and proposes a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issues regarding data sharing in multi-party-owned networked grids, and (2) transfers learned controls from simulation to hardware-in-the-loop test-bed, thereby bridging the gap between simulation and real world. With these multi-prong objectives, first, we formulate a reinforcement learning (RL) training setup generating episodic trajectories with adversaries (attack signal) injected at the primary controllers of the grid forming (GFM) inverters where RL agents (or controllers) are being trained to mitigate the injected attacks. For networked microgrids, the horizontal Fed-RL method involving distinct independent environments is not appropriate, leading us to develop vertical variant Federated Soft Actor-Critic (FedSAC) algorithm to grasp the interconnected dynamics of networked microgrid. Next, utilizing OpenAI Gym interface, we built a custom simulation set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark test systems comprising 3 interconnected microgrids. Finally, the learned policies in simulation world are transferred to the real-time hardware-in-the-loop test-bed set-up developed using high-fidelity Hypersim platform. Experiments show that the simulator-trained RL controllers produce convincing results with the real-time test-bed set-up, validating the minimization of sim-to-real gap.

Empowering the Grid: Decentralized Autonomous Control for Effective Utilization and Resilience

Adaptive Resilient Operation of Cooperative Grid-Forming Converters under Cyber Attacks

A Decentralized Resilient Control Scheme for DC Microgrids Against Faults on Sensor and Actuator

Autonomous Control of Inverters in Microgrid

Decentralized High-Performance Control of DC Microgrids

A FDI Attack-Resilient Distributed Secondary Control Strategy for Islanded Microgrids

Adaptive Decentralized Output-Constrained Control of Single-Bus DC Microgrids

Analysis of Grid-Forming Inverter Controls for Grid-Connected and Islanded Microgrid Integration

A Decentralized Fault-Tolerant Control for DC Microgrids Against Disturbances and Actuator Faults

Resilient Synchronization Strategy for AC Microgrids under Cyber Attacks

Robust Decentralized Secondary Control Scheme for Inverter-based Power Networks

The Microgrid Controller: An Empowering Technology at the Grid Edge: Realizing Benefits From Microgrids Via Sequential De-Risking of Controller Technology

Decentralized Secondary Frequency Control of Autonomous Microgrids Via Adaptive Robust -Gain Performance

Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Grid-Forming Inverters in a Microgrid: Maintaining Power During an Outage and Restoring Connection to the Utility Grid Without Communication

Decentralized Secondary Frequency Control of Autonomous Microgrids via Adaptive Robust L2-Gain Performance

Distributed Cooperative Optimal Control Architecture for AC Microgrid with Renewable Generation and Storage

Auxiliary Network-Enabled Attack Detection and Resilient Control of Islanded AC Microgrid

A Micro-Grid Distributed Intelligent Control and Management System

Nonlinear control of an AC-connected DC MicroGrid