Abstract:Multi-building multi-energy virtual power plants (MB-ME-VPPs) show great promise for the aggregation and coordination of distributed flexible resources across multiple integrated energy buildings to participate in electricity markets. However, a significant challenge arises when managing the energy of MB-ME-VPPs, especially since buildings can dynamically join or depart during the aggregation phase. Traditional model- based optimization methods face difficulties in obtaining accurate mathematical models of individual buildings, and may also raise privacy concerns. In contrast, model-free multi-agent reinforcement learning (MARL) methods offer a promising alternative by allowing agents to learn their control policies through interactions with their environments. Nevertheless, conventional MARL methods are normally applied in static multi- agent environments, where the number and identity of agents remain fixed and predetermined. Consequently, these conventional MARL methods lack the ability to adapt to the dynamic behaviors of agents joining or leaving the environment. To this end, this paper proposes a novel approach named MAT-Adapt, embedding the multi-agent transformer with a parallel adapter module, to address the dynamic participation issue in MB-ME-VPP energy management. Firstly, it formulates the coordination of building agents as a sequential modeling process and leverages the representational capabilities of the attention mechanism from the multi- agent transformer technique. Secondly, it introduces a parallel adapter module called AdaptMLP to enhance adaptability during the dynamic participation phase, efficiently reducing the need for extensive fine-tuning of model parameters. Simulations on the IEEE 33-bus distributional electricity market with 3 to 9 multi- energy buildings show the superior performance of our proposed MAT-Adapt method in facilitating efficient coordination of dynamically participating buildings within the context of the MB-ME-VPP. In comparison to the conventional MADDPG and MAPPO methods training from scratch, the proposed MAT-Adapt method demonstrates its superior adaptability, achieving 0.75-0.91 normalized rewards in new state conditions within 5% of training episodes, while MADDPG and MAPPO can only reach 0.11-0.43 within the same timeframe. Furthermore, the proposed MAT-Adapt method exhibits its strong generalization performance by evaluating the dynamic participation of various building types and regions.

Suppressing Active Power Fluctuations at PCC in Grid-Connection Microgrids Via Multiple BESSs: A Collaborative Multi-Agent Reinforcement Learning Approach

Multi-Agent System Based Optimal Power Dispatch Algorithm for Microgrid

Adaptive Multi-Agent Reinforcement Learning for Flexible Resource Management in a Virtual Power Plant with Dynamic Participating Multi-Energy Buildings

A Cooperative Dispatch Algorithm for Hydrogen-Based Grid-Connection Microgrids: A Multi-Agent Reinforcement Learning Method

Distributed Reactive Power Control and SOC Sharing Method for Battery Energy Storage System in Microgrids

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Multiagent-Based Reinforcement Learning for Optimal Reactive Power Dispatch.

Towards Microgrid Resilience Enhancement Via Mobile Power Sources and Repair Crews: A Multi-Agent Reinforcement Learning Approach

Physics-Shielded Multi-Agent Deep Reinforcement Learning for Safe Active Voltage Control with Photovoltaic/Battery Energy Storage Systems

Physics-Guided Multi-Agent Adversarial Reinforcement Learning for Robust Active Voltage Control With Peer-to-Peer (P2P) Energy Trading

Peer-to-Peer Energy Trading of Solar and Energy Storage: A Networked Multiagent Reinforcement Learning Approach

Hierarchical multi-agent reinforcement learning for repair crews dispatch control towards multi-energy microgrid resilience

Microgrid Optimization Strategy for Charging and Swapping Power Stations with New Energy Based on Multi-Agent Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Photovoltaics and Battery Storage Aggregators Coordinated Operation in Active Distribution Network with Incomplete Information

Stochastic dispatch of energy storage in microgrids: An augmented reinforcement learning approach

Stochastic Dispatch of Energy Storage in Microgrids: A Reinforcement Learning Approach Incorporated with MCTS

PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control

Collaborative Optimization of Multi-microgrids System with Shared Energy Storage Based on Multi-agent Stochastic Game and Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

QMIX-based Multi-Agent Reinforcement Learning for Electric Vehicle-Facilitated Peak Shaving

Multi-energy Management of Interconnected Multi-microgrid System Using Multi-agent Deep Reinforcement Learning