Abstract:Reliability is a fundamental requirement of power systems. However, uncertainties from renewable energy generators and demand loads bring challenges to the economical and reliable operation of power distribution networks. This paper focuses on an energy management problem for networked microgrid systems (NMSs), aiming at establishing energy management policies for individual microgrids within NMSs to enhance the long-term economy and stability of the entire system. The key challenge is that each microgrid has independent decision-making capability and how to collaborate all microgrids’ decisions in a distributed multi-agent manner such that the entire system goal is optimized. We introduce the mean and variance of the exchanged power between the NMS and the main grid as the indicators of NMS operation profits and power fluctuations, respectively. The energy management problem is formulated as a mean-variance team stochastic game (MV-TSG) model. Because the variance metric is neither additive nor Markovian in a dynamic setting, the dynamic programming principle does not hold for this problem. We analyze the MV-TSG from a view of sensitivity-based optimization and propose a multi-agent policy iteration method by introducing a sequential update scheme. Furthermore, to tackle MV-TSGs with larger scales or unknown environmental parameters, we extend the idea of trust region optimization and develop a novel multi-agent deep reinforcement learning algorithm. The performance of our approaches is verified through numerical experiments using a real dataset in power distribution networks. Note to Practitioners —Energy management is essential for economical and reliable operation of networked microgrid systems (NMSs). NMSs exchange power with the main grid, where positive exchanged power indicates that the NMS is selling power to the main grid and generating profits, while negative exchanged power indicates that the NMS is purchasing power and incurring costs. The long-term average exchanged power between the NMS and the main grid reflects the economical operation of NMSs. However, fluctuations in exchanged power can lead to power quality issues such as frequency deviation and voltage flicker, which affect the reliability of NMS operations. In this paper, we study the economical and reliable operation of NMSs with distributed energy management policies, formulating the problem as a mean-variance team stochastic game (MV-TSG). We propose a multi-agent policy iteration method to address MV-TSGs. We further develop a multi-agent deep reinforcement learning algorithm to tackle MV-TSGs with large scales and in scenarios where environmental models are unknown. Experiments utilizing real data demonstrate that our algorithms effectively promote collaboration among microgrids to control exchanged power. As the weight coefficient increases, power fluctuations are reduced, achieving power smoothing and peak shaving simultaneously in long-term scheduling. Consequently, our algorithms can facilitate the trade-off between economy and reliability in NMS operations. These results provide valuable insights into the energy management challenges of NMSs in power distribution networks.

Multi-Agent Learning-Based Nearly Non-Iterative Stochastic Dynamic Transactive Energy Control of Networked Microgrids

Distributed Energy Management for the Multi-Microgrid System Based on ADMM

Distributed Real-Time Energy Management of Multi-microgrid for Incomplete Information Scenarios

Distributed Online Learning and Dynamic Robust Standby Dispatch for Networked Microgrids

A Transactive Energy Framework for Coordinated Energy Management of Networked Microgrids With Distributionally Robust Optimization

Optimal Energy Management for Multi-Microgrid Under a Transactive Energy Framework With Distributionally Robust Optimization

Adaptively Constrained Stochastic Model Predictive Control for the Optimal Dispatch of Microgrid

Economical and Reliable Energy Management for Networked Microgrids in a Multi-Agent Collaborative Manner

Optimal Energy Management of Multi-Microgrids Connected to Distribution System Based on Deep Reinforcement Learning

Multi-energy Management of Interconnected Multi-microgrid System Using Multi-agent Deep Reinforcement Learning

Multi-Microgrid Collaborative Optimization Scheduling Using an Improved Multi-Agent Soft Actor-Critic Algorithm

Coordinated energy management of networked Microgrids in distribution systems

Collaborative Optimization of Multi-microgrids System with Shared Energy Storage Based on Multi-agent Stochastic Game and Reinforcement Learning

Coordinated Energy Scheduling for Autonomous Microgrids Based on Multi-layer Control Architecture

Coordinated Energy Scheduling of a Distributed Multi-Microgrid System Based on Multi-Agent Decisions

Coordinated Energy Dispatch of Autonomous Microgrids with Distributed MPC Optimization

Coordinating Multi-Energy Microgrids for Integrated Energy System Resilience: A Multi-Task Learning Approach

Distributed Stochastic Model Predictive Control for Scheduling Deterministic Peer-to-Peer Energy Transactions Among Networked Microgrids With Hybrid Energy Storage Systems

Deep Reinforcement Learning Based Optimal Energy Management of Multi-energy Microgrids with Uncertainties

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Robust Energy Management for Multi-Microgrids Based on Distributed Dynamic Tube Model Predictive Control