Abstract:Multi-building multi-energy virtual power plants (MB-ME-VPPs) show great promise for the aggregation and coordination of distributed flexible resources across multiple integrated energy buildings to participate in electricity markets. However, a significant challenge arises when managing the energy of MB-ME-VPPs, especially since buildings can dynamically join or depart during the aggregation phase. Traditional model- based optimization methods face difficulties in obtaining accurate mathematical models of individual buildings, and may also raise privacy concerns. In contrast, model-free multi-agent reinforcement learning (MARL) methods offer a promising alternative by allowing agents to learn their control policies through interactions with their environments. Nevertheless, conventional MARL methods are normally applied in static multi- agent environments, where the number and identity of agents remain fixed and predetermined. Consequently, these conventional MARL methods lack the ability to adapt to the dynamic behaviors of agents joining or leaving the environment. To this end, this paper proposes a novel approach named MAT-Adapt, embedding the multi-agent transformer with a parallel adapter module, to address the dynamic participation issue in MB-ME-VPP energy management. Firstly, it formulates the coordination of building agents as a sequential modeling process and leverages the representational capabilities of the attention mechanism from the multi- agent transformer technique. Secondly, it introduces a parallel adapter module called AdaptMLP to enhance adaptability during the dynamic participation phase, efficiently reducing the need for extensive fine-tuning of model parameters. Simulations on the IEEE 33-bus distributional electricity market with 3 to 9 multi- energy buildings show the superior performance of our proposed MAT-Adapt method in facilitating efficient coordination of dynamically participating buildings within the context of the MB-ME-VPP. In comparison to the conventional MADDPG and MAPPO methods training from scratch, the proposed MAT-Adapt method demonstrates its superior adaptability, achieving 0.75-0.91 normalized rewards in new state conditions within 5% of training episodes, while MADDPG and MAPPO can only reach 0.11-0.43 within the same timeframe. Furthermore, the proposed MAT-Adapt method exhibits its strong generalization performance by evaluating the dynamic participation of various building types and regions.

A Novel Multi-agent Cooperative Reinforcement Learning Method for Home Energy Management under a Peak Power-limiting

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Adaptive Multi-Agent Reinforcement Learning for Flexible Resource Management in a Virtual Power Plant with Dynamic Participating Multi-Energy Buildings

A Real-time Demand Response Strategy of Home Energy Management by Using Distributed Deep Reinforcement Learning.

Large-Scale Home Energy Management Using Entropy-Based Collective Multiagent Deep Reinforcement Learning Framework

Multi-agent DRL-based Data-Driven Approach for PEVs Charging/discharging Scheduling in Smart Grid.

Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management

Home Energy Management System Optimization Strategy Based on Reinforcement Learning

Two-timescale autonomous energy management strategy based on multi-agent deep reinforcement learning approach for residential multicarrier energy system

Scalable Coordinated Management of Peer-to-peer Energy Trading: A Multi-Cluster Deep Reinforcement Learning Approach

A Multi-Agent Deep Constrained Q-Learning Method for Smart Building Energy Management Under Uncertainties

Real-time Energy Management of a Low-Carbon Micro-Grid Based on Deep Reinforcement Learning

Interior-point policy optimization based multi-agent deep reinforcement learning method for secure home energy management under various uncertainties

Multi-Agent Reinforcement Learning for Smart Community Energy Management

Privacy-Preserving Load Scheduling in Residential Microgrids Using Multi-Agent Reinforcement Learning

Continuous Multiagent Control Using Collective Behavior Entropy for Large-Scale Home Energy Management

Intelligent Residential Energy Management System using Deep Reinforcement Learning

Demand-Side Scheduling Based on Multi-Agent Deep Actor-Critic Learning for Smart Grids

A Novel Forecasting Based Scheduling Method for Household Energy Management System Based on Deep Reinforcement Learning