Abstract:Multi-building multi-energy virtual power plants (MB-ME-VPPs) show great promise for the aggregation and coordination of distributed flexible resources across multiple integrated energy buildings to participate in electricity markets. However, a significant challenge arises when managing the energy of MB-ME-VPPs, especially since buildings can dynamically join or depart during the aggregation phase. Traditional model- based optimization methods face difficulties in obtaining accurate mathematical models of individual buildings, and may also raise privacy concerns. In contrast, model-free multi-agent reinforcement learning (MARL) methods offer a promising alternative by allowing agents to learn their control policies through interactions with their environments. Nevertheless, conventional MARL methods are normally applied in static multi- agent environments, where the number and identity of agents remain fixed and predetermined. Consequently, these conventional MARL methods lack the ability to adapt to the dynamic behaviors of agents joining or leaving the environment. To this end, this paper proposes a novel approach named MAT-Adapt, embedding the multi-agent transformer with a parallel adapter module, to address the dynamic participation issue in MB-ME-VPP energy management. Firstly, it formulates the coordination of building agents as a sequential modeling process and leverages the representational capabilities of the attention mechanism from the multi- agent transformer technique. Secondly, it introduces a parallel adapter module called AdaptMLP to enhance adaptability during the dynamic participation phase, efficiently reducing the need for extensive fine-tuning of model parameters. Simulations on the IEEE 33-bus distributional electricity market with 3 to 9 multi- energy buildings show the superior performance of our proposed MAT-Adapt method in facilitating efficient coordination of dynamically participating buildings within the context of the MB-ME-VPP. In comparison to the conventional MADDPG and MAPPO methods training from scratch, the proposed MAT-Adapt method demonstrates its superior adaptability, achieving 0.75-0.91 normalized rewards in new state conditions within 5% of training episodes, while MADDPG and MAPPO can only reach 0.11-0.43 within the same timeframe. Furthermore, the proposed MAT-Adapt method exhibits its strong generalization performance by evaluating the dynamic participation of various building types and regions.

MARL-iDR: Multi-Agent Reinforcement Learning for Incentive-based Residential Demand Response

Resilience Enhancement of Multi-Agent Reinforcement Learning-Based Demand Response Against Adversarial Attacks

Cooperative price-based demand response program for multiple aggregators based on multi-agent reinforcement learning and shapley-value

Privacy-preserving Demand Response of Aggregated Residential Load

Demand Response Management with Multiple Utility Companies: A Two-Level Game Approach.

Agent-based Model for Electricity Consumption and Storage to Evaluate Economic Viability of Tariff Arbitrage for Residential Sector Demand Response

Adaptive Multi-Agent Reinforcement Learning for Flexible Resource Management in a Virtual Power Plant with Dynamic Participating Multi-Energy Buildings

Price-Based Residential Demand Response Management in Smart Grids: A Reinforcement Learning-Based Approach

Incremental Incentive Mechanism Design for Diversified Consumers in Demand Response

Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance

Edge-End Based Demand Response System of Aggregated Residential Load: Case Study

Principal-Agent Model for Bilateral Contract Design to Incentivize Residential Demand-Side Flexibility

An effective energy management Layout-Based reinforcement learning for household demand response in digital twin simulation

User Preference-Based Demand Response for Smart Home Energy Management Using Multiobjective Reinforcement Learning

Modified deep learning and reinforcement learning for an incentive-based demand response model

Peak shaving and self-consumption maximization in home energy management systems: A combined integer programming and reinforcement learning approach

Limiting gaming opportunities on incentive-based demand response programs

Optimal Incentive Strategy in Cloud-Edge Integrated Demand Response Framework for Residential Air Conditioning Loads

Asynchronous Multi-Agent Reinforcement Learning-Based Framework for Bi-Level Noncooperative Game-Theoretic Demand Response

Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework

A Fast Distributed Algorithm for Large-Scale Demand Response Aggregation