Abstract:Multi-building multi-energy virtual power plants (MB-ME-VPPs) show great promise for the aggregation and coordination of distributed flexible resources across multiple integrated energy buildings to participate in electricity markets. However, a significant challenge arises when managing the energy of MB-ME-VPPs, especially since buildings can dynamically join or depart during the aggregation phase. Traditional model- based optimization methods face difficulties in obtaining accurate mathematical models of individual buildings, and may also raise privacy concerns. In contrast, model-free multi-agent reinforcement learning (MARL) methods offer a promising alternative by allowing agents to learn their control policies through interactions with their environments. Nevertheless, conventional MARL methods are normally applied in static multi- agent environments, where the number and identity of agents remain fixed and predetermined. Consequently, these conventional MARL methods lack the ability to adapt to the dynamic behaviors of agents joining or leaving the environment. To this end, this paper proposes a novel approach named MAT-Adapt, embedding the multi-agent transformer with a parallel adapter module, to address the dynamic participation issue in MB-ME-VPP energy management. Firstly, it formulates the coordination of building agents as a sequential modeling process and leverages the representational capabilities of the attention mechanism from the multi- agent transformer technique. Secondly, it introduces a parallel adapter module called AdaptMLP to enhance adaptability during the dynamic participation phase, efficiently reducing the need for extensive fine-tuning of model parameters. Simulations on the IEEE 33-bus distributional electricity market with 3 to 9 multi- energy buildings show the superior performance of our proposed MAT-Adapt method in facilitating efficient coordination of dynamically participating buildings within the context of the MB-ME-VPP. In comparison to the conventional MADDPG and MAPPO methods training from scratch, the proposed MAT-Adapt method demonstrates its superior adaptability, achieving 0.75-0.91 normalized rewards in new state conditions within 5% of training episodes, while MADDPG and MAPPO can only reach 0.11-0.43 within the same timeframe. Furthermore, the proposed MAT-Adapt method exhibits its strong generalization performance by evaluating the dynamic participation of various building types and regions.

Coordinated Energy Management Strategy for Multi-Energy Hub with Thermo-Electrochemical Effect Based Power-to-ammonia: A Multi-Agent Deep Reinforcement Learning Enabled Approach

Adaptive Multi-Agent Reinforcement Learning for Flexible Resource Management in a Virtual Power Plant with Dynamic Participating Multi-Energy Buildings

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Integrated Modelling and Enhanced Utilization of Power-to-Ammonia for High Renewable Penetrated Multi-Energy Systems

Peer-to-Peer Energy Trading and Energy Conversion in Interconnected Multi-Energy Microgrids Using Multi-Agent Deep Reinforcement Learning

A Multi-Agent Deep-Reinforcement-Learning-Based Strategy for Safe Distributed Energy Resource Scheduling in Energy Hubs

Distributed energy management of multi-area integrated energy system based on multi-agent deep reinforcement learning

Two-timescale autonomous energy management strategy based on multi-agent deep reinforcement learning approach for residential multicarrier energy system

Microgrid Optimization Strategy for Charging and Swapping Power Stations with New Energy Based on Multi-Agent Reinforcement Learning

Energy management based on multi-agent deep reinforcement learning for a multi-energy industrial park

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

Distributed deep reinforcement learning-based coordination performance optimization method for proton exchange membrane fuel cell system

Data Driven Solution to Market Equilibrium Via Deep Reinforcement Learning

Intelligent hydrogen-ammonia combined energy storage system with deep reinforcement learning

Peer-to-peer energy trading with energy trading consistency in interconnected multi-energy microgrids: A multi-agent deep reinforcement learning approach

Energy Management of Multi-mode Plug-in Hybrid Electric Vehicle using Multi-agent Deep Reinforcement Learning

Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market

Joint Energy and Carbon Trading for Multi-Microgrid System Based on Multi-Agent Deep Reinforcement Learning

Mean-Field Multi-Agent Reinforcement Learning for Peer-to-Peer Multi-Energy Trading

Optimal Operation of a Hydrogen-based Building Multi-Energy System Based on Deep Reinforcement Learning

Online energy management strategy for ammonia-hydrogen hybrid electric vehicles harnessing deep reinforcement learning