Abstract:Reliability is a fundamental requirement of power systems. However, uncertainties from renewable energy generators and demand loads bring challenges to the economical and reliable operation of power distribution networks. This paper focuses on an energy management problem for networked microgrid systems (NMSs), aiming at establishing energy management policies for individual microgrids within NMSs to enhance the long-term economy and stability of the entire system. The key challenge is that each microgrid has independent decision-making capability and how to collaborate all microgrids’ decisions in a distributed multi-agent manner such that the entire system goal is optimized. We introduce the mean and variance of the exchanged power between the NMS and the main grid as the indicators of NMS operation profits and power fluctuations, respectively. The energy management problem is formulated as a mean-variance team stochastic game (MV-TSG) model. Because the variance metric is neither additive nor Markovian in a dynamic setting, the dynamic programming principle does not hold for this problem. We analyze the MV-TSG from a view of sensitivity-based optimization and propose a multi-agent policy iteration method by introducing a sequential update scheme. Furthermore, to tackle MV-TSGs with larger scales or unknown environmental parameters, we extend the idea of trust region optimization and develop a novel multi-agent deep reinforcement learning algorithm. The performance of our approaches is verified through numerical experiments using a real dataset in power distribution networks. Note to Practitioners —Energy management is essential for economical and reliable operation of networked microgrid systems (NMSs). NMSs exchange power with the main grid, where positive exchanged power indicates that the NMS is selling power to the main grid and generating profits, while negative exchanged power indicates that the NMS is purchasing power and incurring costs. The long-term average exchanged power between the NMS and the main grid reflects the economical operation of NMSs. However, fluctuations in exchanged power can lead to power quality issues such as frequency deviation and voltage flicker, which affect the reliability of NMS operations. In this paper, we study the economical and reliable operation of NMSs with distributed energy management policies, formulating the problem as a mean-variance team stochastic game (MV-TSG). We propose a multi-agent policy iteration method to address MV-TSGs. We further develop a multi-agent deep reinforcement learning algorithm to tackle MV-TSGs with large scales and in scenarios where environmental models are unknown. Experiments utilizing real data demonstrate that our algorithms effectively promote collaboration among microgrids to control exchanged power. As the weight coefficient increases, power fluctuations are reduced, achieving power smoothing and peak shaving simultaneously in long-term scheduling. Consequently, our algorithms can facilitate the trade-off between economy and reliability in NMS operations. These results provide valuable insights into the energy management challenges of NMSs in power distribution networks.

A Multiagent Approach for Modeling Power-Supply Systems with MicroGrid

Multi-Agent System Based Optimal Power Dispatch Algorithm for Microgrid

A Multiagent-Based Game-Theoretic and Optimization Approach for Market Operation of Multimicrogrid Systems

GridLAB-D: An agent-based simulation framework for smart grids

Multi-Agent System with Plug and Play Feature for Distributed Secondary Control in Microgrid—Controller and Power Hardware-in-the-Loop Implementation

Real-Time Multi-Agent Based Power Management of Virtually Integrated Microgrids Comprising Prosumers of Plug-in Electric Vehicles and Renewable Energy Sources

Sustainable Intelligent Energy Management System for Microgrid Using Multi-Agent Systems: A Case Study

A Novel Mathematical Approach to Model Multi-Agent-Based Main Grid and Microgrid Networks for Complete System Analysis

A self-organizing multi-agent system for distributed voltage regulation

Multi-agent system for microgrids: design, optimization and performance

Economical and Reliable Energy Management for Networked Microgrids in a Multi-Agent Collaborative Manner

The Cooperative Control of Distributed Generation Based on Multi-agent System

An Integrated Framework of Agent-Based Modelling and Robust Optimization for Microgrid Energy Management

Energy management and control system for microgrid based wind-PV-battery using multi-agent systems

Microgrid energy management system for smart home using multi-agent system

Reliable microgrid energy management under environmental uncertainty and mechanical failures: An agent-based modeling and robust optimization approach

Optimal energy management and control aspects of distributed microgrid using multi-agent systems

A Multi-Agent Approach to Optimal Sizing of a Combined Heating and Power Microgrid

Development of a Multi-Agent System for Optimal Sizing of a Commercial Complex Microgrid

Multiagent-Based Hybrid Energy Management System for Microgrids

Decentralized Control of the Power Flows in a Network of Smart Microgrids Modeled as a Team of Cooperative Agents