Abstract:While distributed secondary controllers have been studied for multiple energy storage systems in islanded microgrids, information infrastructure has to be added for the extensive information transmission among these secondary controllers and the additional communication among distributed controllers is costly and increases the vulnerability surface to cyberattacks. In this work, a data-driven decentralized secondary control scheme is proposed for multiple heterogeneous battery energy storage systems (BESSs). The proposed secondary control scheme can achieve frequency regulation and the state-of-charge (SoC) balancing simultaneously for BESSs without requiring accurate BESS models. This scheme leverages an asynchronous advantage actor-critic (A3C) based multi-agent deep reinforcement learning (MA-DRL) algorithm where the centralized off-line learning with shared convolutional neural networks (CNN) is designed to maximize global rewards and ensure the performance of the entire system and a decentralized online execution mechanism is applied to each BESS. Furthermore, in view of possible denial-of-service (DoS) attack on local communication networks used for signal transfer between secondary controllers and remote sensors, a signal-to-interference-plus-noise ratio (SINR)-based dynamic and proactive event-triggered communication mechanism is proposed to alleviate the impact of DoS attacks and reduce the occupation of communication resources. Simulation results on a four-bus multiple BESS system show that the proposed decentralized secondary controller can achieve simultaneous frequency regulation and SoC balancing. Comparison results with other event-triggered mechanisms and MA-DRL algorithms show the A3C based MA-DRL algorithm with CNN can obtain a comparatively optimal policy through training and the designed event-triggered strategy can dynamically adapt the release frequency based on real-time SINR and significantly reduce the occupied network bandwidth and packet loss rate (PER) induced by DoS attacks.

Multi-Agent Reinforcement Learning for Decentralized Resilient Secondary Control of Energy Storage Systems Against DoS Attacks

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Secondary Voltage Collaborative Control of Distributed Energy System via Multi-Agent Reinforcement Learning

Multiagent Distributed Secondary Control for Energy Storage Systems With Lossy Communication Networks in DC Microgrid

Data-Based Collaborative Learning for Multiagent Systems under Distributed Denial-of-Service Attacks

Decentralized Multi-agent Reinforcement Learning based State-of-Charge Balancing Strategy for Distributed Energy Storage System

Distributed consensus controlled multi-battery-energy-storage-system under denial-of-service attacks

Event-Triggered Secure Consensus Control for DC Microgrids under DoS Attacks

Optimal Distributed Coordinated Reinforcement Learning for Secondary Voltage Control in Time-Delayed Microgrid

Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization

Data-driven load frequency cooperative control for multi-area power system integrated with VSCs and EV aggregators under cyber-attacks

A Safe Policy Learning-Based Method for Decentralized and Economic Frequency Control in Isolated Networked-Microgrid Systems

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

Physics-Shielded Multi-Agent Deep Reinforcement Learning for Safe Active Voltage Control with Photovoltaic/Battery Energy Storage Systems

Coordinated Dispatch of Energy Storage Systems in the Active Distribution Network: A Complementary Reinforcement Learning and Optimization Approach

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

Distributed Resilient Secondary Control for Microgrids with Attention-based Weights against High-density Misbehaving Agents

Energy-Management Strategy of Battery Energy Storage Systems in DC Microgrids: A Distributed Fuzzy Output Consensus Control Considering Multiple Cyber Attacks

Decentralized Graphical-Representation-Enabled Multi-Agent Deep Reinforcement Learning for Robust Control of Cyber-Physical Systems

A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment