Abstract:Multi-building multi-energy virtual power plants (MB-ME-VPPs) show great promise for the aggregation and coordination of distributed flexible resources across multiple integrated energy buildings to participate in electricity markets. However, a significant challenge arises when managing the energy of MB-ME-VPPs, especially since buildings can dynamically join or depart during the aggregation phase. Traditional model- based optimization methods face difficulties in obtaining accurate mathematical models of individual buildings, and may also raise privacy concerns. In contrast, model-free multi-agent reinforcement learning (MARL) methods offer a promising alternative by allowing agents to learn their control policies through interactions with their environments. Nevertheless, conventional MARL methods are normally applied in static multi- agent environments, where the number and identity of agents remain fixed and predetermined. Consequently, these conventional MARL methods lack the ability to adapt to the dynamic behaviors of agents joining or leaving the environment. To this end, this paper proposes a novel approach named MAT-Adapt, embedding the multi-agent transformer with a parallel adapter module, to address the dynamic participation issue in MB-ME-VPP energy management. Firstly, it formulates the coordination of building agents as a sequential modeling process and leverages the representational capabilities of the attention mechanism from the multi- agent transformer technique. Secondly, it introduces a parallel adapter module called AdaptMLP to enhance adaptability during the dynamic participation phase, efficiently reducing the need for extensive fine-tuning of model parameters. Simulations on the IEEE 33-bus distributional electricity market with 3 to 9 multi- energy buildings show the superior performance of our proposed MAT-Adapt method in facilitating efficient coordination of dynamically participating buildings within the context of the MB-ME-VPP. In comparison to the conventional MADDPG and MAPPO methods training from scratch, the proposed MAT-Adapt method demonstrates its superior adaptability, achieving 0.75-0.91 normalized rewards in new state conditions within 5% of training episodes, while MADDPG and MAPPO can only reach 0.11-0.43 within the same timeframe. Furthermore, the proposed MAT-Adapt method exhibits its strong generalization performance by evaluating the dynamic participation of various building types and regions.

Attention-Enhanced Multi-Agent Reinforcement Learning Against Observation Perturbations for Distributed Volt-VAR Control

Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

Multi-agent Graph Reinforcement Learning for Decentralized Volt-VAR Control in Power Distribution Systems

Online Multi-Agent Reinforcement Learning for Decentralized Inverter-Based Volt-VAR Control

Adaptive Multi-Agent Reinforcement Learning for Flexible Resource Management in a Virtual Power Plant with Dynamic Participating Multi-Energy Buildings

Multi-Agent Reinforcement Learning-based Distributed Economic Dispatch Considering Network attacks and Uncertain Costs

Volt-VAR Control in Active Distribution Networks Using Multi-Agent Reinforcement Learning

Robust Regional Coordination of Inverter-Based Volt/Var Control Via Multi-Agent Deep Reinforcement Learning

Reinforcement Learning Based Robust Volt/Var Control in Active Distribution Networks With Imprecisely Known Delay

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

Multi Agent Safe Graph Reinforcement Learning for PV Inverter s Based Real-Time De centralized Volt/Var Control in Zoned Distribution Networks

Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control

Bi-level Off-policy Reinforcement Learning for Two-Timescale Volt/VAR Control in Active Distribution Networks

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks

Decentralized Graphical-Representation-Enabled Multi-Agent Deep Reinforcement Learning for Robust Control of Cyber-Physical Systems

Secondary Voltage Collaborative Control of Distributed Energy System via Multi-Agent Reinforcement Learning

Energy Management Based on Safe Multi-Agent Reinforcement Learning for Smart Buildings in Distribution Networks

Multi-Agent Reinforcement Learning with Safety Layer for Active Voltage Control

Robust Data-driven and Fully Distributed Volt/VAR Control for Active Distribution Networks with Multiple Virtual Power Plants

Bi-level Off-policy Reinforcement Learning for Volt/VAR Control Involving Continuous and Discrete Devices