Abstract:While distributed secondary controllers have been studied for multiple energy storage systems in islanded microgrids, information infrastructure has to be added for the extensive information transmission among these secondary controllers and the additional communication among distributed controllers is costly and increases the vulnerability surface to cyberattacks. In this work, a data-driven decentralized secondary control scheme is proposed for multiple heterogeneous battery energy storage systems (BESSs). The proposed secondary control scheme can achieve frequency regulation and the state-of-charge (SoC) balancing simultaneously for BESSs without requiring accurate BESS models. This scheme leverages an asynchronous advantage actor-critic (A3C) based multi-agent deep reinforcement learning (MA-DRL) algorithm where the centralized off-line learning with shared convolutional neural networks (CNN) is designed to maximize global rewards and ensure the performance of the entire system and a decentralized online execution mechanism is applied to each BESS. Furthermore, in view of possible denial-of-service (DoS) attack on local communication networks used for signal transfer between secondary controllers and remote sensors, a signal-to-interference-plus-noise ratio (SINR)-based dynamic and proactive event-triggered communication mechanism is proposed to alleviate the impact of DoS attacks and reduce the occupation of communication resources. Simulation results on a four-bus multiple BESS system show that the proposed decentralized secondary controller can achieve simultaneous frequency regulation and SoC balancing. Comparison results with other event-triggered mechanisms and MA-DRL algorithms show the A3C based MA-DRL algorithm with CNN can obtain a comparatively optimal policy through training and the designed event-triggered strategy can dynamically adapt the release frequency based on real-time SINR and significantly reduce the occupied network bandwidth and packet loss rate (PER) induced by DoS attacks.

Distributed Actor-Critic Neural Networks-Based Structure for Frequency Synchronization in Isolated AC Microgrids

Robust and Resilient Distributed Optimal Frequency Control for Microgrids Against Cyber Attacks

Distributed Optimal Tie-Line Power Flow Control for Multiple Interconnected AC Microgrids

Distributed Optimal Synchronization Rate Control for AC Microgrids Under Event-Triggered Mechanism

A Safe Policy Learning-Based Method for Decentralized and Economic Frequency Control in Isolated Networked-Microgrid Systems

Distributed Finite-Time Control of Islanded Microgrid for Ancillary Services Provision

Optimal Distributed Cooperative Control Strategy for Frequency Restoration in AC Microgrid under Malicious Attacks

Distributed Hybrid Secondary Control of Virtual Synchronous Generators for Isolated AC Microgrids with Low Bandwidth Communication

Deep reinforcement learning for adaptive frequency control of island microgrid considering control performance and economy

Complex network control and stability through distributed critic‐based neuro‐fuzzy learning

A novel reinforcement learning policy optimization based adaptive VSG control technique for improved frequency stabilization in AC microgrids

Generating adversarial deep reinforcement learning -based frequency control of Island City microgrid considering generalization of scenarios

Distributed synchronization based on model-free reinforcement learning in wireless ad hoc networks

Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guarantees

Coordinated Frequency Control for Multiple Microgrids in Energy Internet: A Stochastic H∞ Approach

Synchronous oscillation characteristic and finite-time function projection synchronization control method for microgrids

Deep Reinforcement Learning for Load Frequency Control in Isolated Microgrids: A Knowledge Aggregation Approach with Emphasis on Power Symmetry and Balance

Distributed Weight Adaptive Control for Frequency Regulation of Islanded Microgrid

AC/DC hybrid distribution network reconfiguration with microgrid formation using multi-agent soft actor-critic

Stable Reinforcement Learning for Optimal Frequency Control: A Distributed Averaging-Based Integral Approach

Multi-Agent Reinforcement Learning for Decentralized Resilient Secondary Control of Energy Storage Systems Against DoS Attacks