Abstract:Multi-building multi-energy virtual power plants (MB-ME-VPPs) show great promise for the aggregation and coordination of distributed flexible resources across multiple integrated energy buildings to participate in electricity markets. However, a significant challenge arises when managing the energy of MB-ME-VPPs, especially since buildings can dynamically join or depart during the aggregation phase. Traditional model- based optimization methods face difficulties in obtaining accurate mathematical models of individual buildings, and may also raise privacy concerns. In contrast, model-free multi-agent reinforcement learning (MARL) methods offer a promising alternative by allowing agents to learn their control policies through interactions with their environments. Nevertheless, conventional MARL methods are normally applied in static multi- agent environments, where the number and identity of agents remain fixed and predetermined. Consequently, these conventional MARL methods lack the ability to adapt to the dynamic behaviors of agents joining or leaving the environment. To this end, this paper proposes a novel approach named MAT-Adapt, embedding the multi-agent transformer with a parallel adapter module, to address the dynamic participation issue in MB-ME-VPP energy management. Firstly, it formulates the coordination of building agents as a sequential modeling process and leverages the representational capabilities of the attention mechanism from the multi- agent transformer technique. Secondly, it introduces a parallel adapter module called AdaptMLP to enhance adaptability during the dynamic participation phase, efficiently reducing the need for extensive fine-tuning of model parameters. Simulations on the IEEE 33-bus distributional electricity market with 3 to 9 multi- energy buildings show the superior performance of our proposed MAT-Adapt method in facilitating efficient coordination of dynamically participating buildings within the context of the MB-ME-VPP. In comparison to the conventional MADDPG and MAPPO methods training from scratch, the proposed MAT-Adapt method demonstrates its superior adaptability, achieving 0.75-0.91 normalized rewards in new state conditions within 5% of training episodes, while MADDPG and MAPPO can only reach 0.11-0.43 within the same timeframe. Furthermore, the proposed MAT-Adapt method exhibits its strong generalization performance by evaluating the dynamic participation of various building types and regions.

MARLYC: Multi-Agent Reinforcement Learning Yaw Control

Adaptive Multi-Agent Reinforcement Learning for Flexible Resource Management in a Virtual Power Plant with Dynamic Participating Multi-Energy Buildings

Skip Training for Multi-Agent Reinforcement Learning Controller for Industrial Wave Energy Converters

An Improved Yaw Control Algorithm for Wind Turbines via Reinforcement Learning

A Multiagent Reinforcement Learning Approach for Wind Farm Frequency Control

Reinforcement learning to maximize wind turbine energy generation

Reinforcement learning to maximise wind turbine energy generation

Collective Large-scale Wind Farm Multivariate Power Output Control Based on Hierarchical Communication Multi-Agent Proximal Policy Optimization

Reinforcement Learning-Based Wind Farm Control: Toward Large Farm Applications via Automatic Grouping and Transfer Learning

Coordinated control of wind turbine and hybrid energy storage system based on multi-agent deep reinforcement learning for wind power smoothing

A grouping strategy for reinforcement learning-based collective yaw control of wind farms

Reinforcement Learning-Based Multiobjective Control of Grid-Connected Wind Farms

Multi-Objective Offshore Wind Farm Wake Redirection Optimization for Power Maximization and Load Reduction

Optimizing Wind Farm Efficiency through Active Yaw Control: A Neural Network-Aided Game Theory Approach

Evaluation of LES-based time-decoupled model-predictive control in different wind farm layouts

Deep Reinforcement Learning for Multi-Objective Optimization: Enhancing Wind Turbine Energy Generation while Mitigating Noise Emissions

Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves

Fuzzy-Based Collective Pitch Control for Wind Turbine via Deep Reinforcement Learning

Multi-Agent Reinforcement Learning for the Low-Level Control of a Quadrotor UAV

Multi-Agent Reinforcement Learning for Power System Operation and Control

Scalable Optimization for Wind Farm Control using Coordination Graphs