Abstract:The utilization of large-scale distributed renewable energy promotes the development of the multi-microgrid (MMG), which raises the need of developing an effective energy management method to minimize economic costs and keep self energy-sufficiency. The multi-agent deep reinforcement learning (MADRL) has been widely used for the energy management problem because of its real-time scheduling ability. However, its training requires massive energy operation data of microgrids (MGs), while gathering these data from different MGs would threaten their privacy and data security. Therefore, this paper tackles this practical yet challenging issue by proposing a federated multi-agent deep reinforcement learning (F-MADRL) algorithm via the physics-informed reward. In this algorithm, the federated learning (FL) mechanism is introduced to train the F-MADRL algorithm thus ensures the privacy and the security of data. In addition, a decentralized MMG model is built, and the energy of each participated MG is managed by an agent, which aims to minimize economic costs and keep self energy-sufficiency according to the physics-informed reward. At first, MGs individually execute the self-training based on local energy operation data to train their local agent models. Then, these local models are periodically uploaded to a server and their parameters are aggregated to build a global agent, which will be broadcasted to MGs and replace their local agents. In this way, the experience of each MG agent can be shared and the energy operation data is not explicitly transmitted, thus protecting the privacy and ensuring data security. Finally, experiments are conducted on Oak Ridge national laboratory distributed energy control communication lab microgrid (ORNL-MG) test system, and the comparisons are carried out to verify the effectiveness of introducing the FL mechanism and the outperformance of our proposed F-MADRL.

Privacy-Preserving Load Scheduling in Residential Microgrids Using Multi-Agent Reinforcement Learning

Privacy Preserving Load Control of Residential Microgrid Via Deep Reinforcement Learning

Privacy Preserving Demand Side Management Method Via Multi-Agent Reinforcement Learning

Cooperative Multi-Agent Actor-Critic for Privacy-Preserving Load Scheduling in a Residential Microgrid

Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids

Federated Multi-Agent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multi-Microgrid Energy Management

Federated Multiagent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multimicrogrid Energy Management

Federated Reinforcement Learning for smart and privacy-preserving energy management of residential microgrids clusters

A Scalable Privacy-Preserving Multi-Agent Deep Reinforcement Learning Approach for Large-Scale Peer-to-Peer Transactive Energy Trading

Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance

Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling

A Novel Multi-agent Cooperative Reinforcement Learning Method for Home Energy Management under a Peak Power-limiting

Deep Reinforcement Learning Based Bi-layer Optimal Scheduling for Microgrids Considering Flexible Load Control

Distributed Energy Trading and Scheduling Among Microgrids via Multiagent Reinforcement Learning

Physical Informed-Inspired Deep Reinforcement Learning Based Bi-Level Programming for Microgrid Scheduling

Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization

Proactive Load-Shaping Strategies with Privacy-Cost Trade-offs in Residential Households based on Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

Optimal Privacy-Preserving Load Scheduling in Smart Grid

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Multi-agent DRL-based Data-Driven Approach for PEVs Charging/discharging Scheduling in Smart Grid.