Abstract:The utilization of large-scale distributed renewable energy promotes the development of the multi-microgrid (MMG), which raises the need of developing an effective energy management method to minimize economic costs and keep self energy-sufficiency. The multi-agent deep reinforcement learning (MADRL) has been widely used for the energy management problem because of its real-time scheduling ability. However, its training requires massive energy operation data of microgrids (MGs), while gathering these data from different MGs would threaten their privacy and data security. Therefore, this paper tackles this practical yet challenging issue by proposing a federated multi-agent deep reinforcement learning (F-MADRL) algorithm via the physics-informed reward. In this algorithm, the federated learning (FL) mechanism is introduced to train the F-MADRL algorithm thus ensures the privacy and the security of data. In addition, a decentralized MMG model is built, and the energy of each participated MG is managed by an agent, which aims to minimize economic costs and keep self energy-sufficiency according to the physics-informed reward. At first, MGs individually execute the self-training based on local energy operation data to train their local agent models. Then, these local models are periodically uploaded to a server and their parameters are aggregated to build a global agent, which will be broadcasted to MGs and replace their local agents. In this way, the experience of each MG agent can be shared and the energy operation data is not explicitly transmitted, thus protecting the privacy and ensuring data security. Finally, experiments are conducted on Oak Ridge national laboratory distributed energy control communication lab microgrid (ORNL-MG) test system, and the comparisons are carried out to verify the effectiveness of introducing the FL mechanism and the outperformance of our proposed F-MADRL.

Federated Reinforcement Learning for Energy Management of Multiple Smart Homes With Distributed Energy Resources

A Federated DRL Approach for Smart Micro-Grid Energy Control with Distributed Energy Resources

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

Federated Deep Q-Network for Multiple Microgrids Energy Management

Federated Multiagent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multimicrogrid Energy Management

Federated Multi-Agent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multi-Microgrid Energy Management

Deep reinforcement learning based home energy management system with devices operational dependencies

Adaptive personalized federated reinforcement learning for multiple-ESS optimal market dispatch strategy with electric vehicles and photovoltaic power generations

Federated Reinforcement Learning for smart and privacy-preserving energy management of residential microgrids clusters

A Hierarchical Deep Reinforcement Learning-Based Community Energy Trading Scheme for a Neighborhood of Smart Households

Reinforcement Learning-Based Energy Management of Smart Home with Rooftop Solar Photovoltaic System, Energy Storage System, and Home Appliances

Federated learning assisted distributed energy optimization

An effective energy management Layout-Based reinforcement learning for household demand response in digital twin simulation

Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning

Robust Federated Deep Reinforcement Learning for Optimal Control in Multiple Virtual Power Plants with Electric Vehicles

Federated Dueling DQN Based Microgrid Energy Management Strategy in Edge-Cloud Computing Environment

A Novel Model-Free Deep Reinforcement Learning Framework for Energy Management of a PV Integrated Energy Hub

DRL-HEMS: Deep Reinforcement Learning Agent for Demand Response in Home Energy Management Systems Considering Customers and Operators Perspectives

Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids

Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework

Federated Learning for Distributed Energy-Efficient Resource Allocation