Abstract:While distributed secondary controllers have been studied for multiple energy storage systems in islanded microgrids, information infrastructure has to be added for the extensive information transmission among these secondary controllers and the additional communication among distributed controllers is costly and increases the vulnerability surface to cyberattacks. In this work, a data-driven decentralized secondary control scheme is proposed for multiple heterogeneous battery energy storage systems (BESSs). The proposed secondary control scheme can achieve frequency regulation and the state-of-charge (SoC) balancing simultaneously for BESSs without requiring accurate BESS models. This scheme leverages an asynchronous advantage actor-critic (A3C) based multi-agent deep reinforcement learning (MA-DRL) algorithm where the centralized off-line learning with shared convolutional neural networks (CNN) is designed to maximize global rewards and ensure the performance of the entire system and a decentralized online execution mechanism is applied to each BESS. Furthermore, in view of possible denial-of-service (DoS) attack on local communication networks used for signal transfer between secondary controllers and remote sensors, a signal-to-interference-plus-noise ratio (SINR)-based dynamic and proactive event-triggered communication mechanism is proposed to alleviate the impact of DoS attacks and reduce the occupation of communication resources. Simulation results on a four-bus multiple BESS system show that the proposed decentralized secondary controller can achieve simultaneous frequency regulation and SoC balancing. Comparison results with other event-triggered mechanisms and MA-DRL algorithms show the A3C based MA-DRL algorithm with CNN can obtain a comparatively optimal policy through training and the designed event-triggered strategy can dynamically adapt the release frequency based on real-time SINR and significantly reduce the occupied network bandwidth and packet loss rate (PER) induced by DoS attacks.

Secondary Voltage Collaborative Control of Distributed Energy System via Multi-Agent Reinforcement Learning

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

Distributed Real-Time Multi-Objective Control of a Virtual Power Plant in DC Distribution Systems

Multiagent Based Distributed Control with Time-Oriented SoC Balancing Method for DC Microgrid

Deep Reinforcement Learning Based Coordinated Voltage Control in Smart Distribution Network

Centralized-Distributed Coordinated Voltage Control of Active Distribution Networks with Renewables

A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control

Multi-Agent Reinforcement Learning for Decentralized Resilient Secondary Control of Energy Storage Systems Against DoS Attacks

Optimal Distributed Coordinated Reinforcement Learning for Secondary Voltage Control in Time-Delayed Microgrid

Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks

Multi-Agent Reinforcement Learning with Safety Layer for Active Voltage Control

Distributed Estimation And Secondary Control Of Autonomous Microgrid

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning-Based Distributed Voltage Control of Flexible Distribution Networks with Soft Open Points

Volt-VAR Control in Active Distribution Networks Using Multi-Agent Reinforcement Learning

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Multi-Agent Q-Value Mixing Network with Covariance Matrix Adaptation Strategy for the Voltage Regulation Problem

Distributed Secondary Frequency and Voltage Control of Multiagent Based Autonomous Micro Grid with Enhanced Active and Reactive Power Sharing

Multi-agent Graph Reinforcement Learning for Decentralized Volt-VAR Control in Power Distribution Systems

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks