Abstract:Residential and commercial buildings are responsible for 35% of the EU energy-related greenhouse gas (GHG) emissions. Reducing their emissions is crucial for meeting the challenging EU objective of the agenda for becoming a net-zero continent by 2050. The diffusion and integration of distributed renewable energy sources (RESs) and energy storage systems (ESSs), as well as the creation of energy communities (ECs), have proven to be crucial aspects in reducing GHG emissions. In this context, this article proposes a multi-agent AI-based control framework to solve the EC's energy management problem in the presence of distributed RESs and ESSs as well as considering a shared ESS. The objectives of the proposed control framework are to satisfy the EC members' load demand to maximize self-consumption and to manage ESSs charging and discharging processes, to enforce cooperative behavior among the EC members by adopting fair and personalized strategies and to maximize EC members' profits. The proposed control procedure is based on three sequential stages, each solved by a dedicated local RL agent exploiting the Q-Learning algorithm. To reduce the computational complexity of the proposed approach, specifically defined state aggregation criteria were defined to map the RL agents' continuous state spaces into discrete state spaces of limited dimensions. During the training phase, the EC members' profiles and the ESSs' and RESs' characteristics were randomly changed to allow the RL agents to learn the correct policy to follow in any given scenario. Simulations proved the effectiveness of the proposed approach for different costumers' load demand profiles and different EC configurations. Indeed, the trained RL agents proved to be able to satisfy the EC members' load demands to maximize self-consumption, to correctly use the distributed and shared ESSs, to charge them according to respective personalized criteria and to sell the energy surplus, prioritizing sales to the EC. The proposed control framework also proved to be a useful tool for understanding EC performance in different configurations and, thus, for properly dimensioning the EC elements.

Fast Stackelberg Equilibrium Learning for Real-Time Coordinated Energy Control of a Multi-Area Integrated Energy System

A Multi-Timescale Allocation Algorithm of Energy and Power for Demand Response in Smart Grids: A Stackelberg Game Approach

Coordinated load frequency control of multi-area integrated energy system using multi-agent deep reinforcement learning

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Low-carbon economic dispatch strategy for integrated electrical and gas system with GCCP based on multi-agent deep reinforcement learning

Integrated Energy Hub Dispatch with a Multi-Mode CAES–BESS Hybrid System: an Option-Based Hierarchical Reinforcement Learning Approach

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

Coordinated automatic generation control of interconnected power system with imitation guided exploration multi-agent deep reinforcement learning

Stackelberg-Nash asymmetric bargaining-based scheduling optimization and revenue-allocation for multi-operator regional integrated energy system considering competition-cooperation relationship and source-load uncertainties

Multiagent deep meta reinforcement learning for sea computing-based energy management of interconnected grids considering renewable energy sources in sustainable cities

Decentralized multi-objective cloud energy storage operation control with deep reinforcement learning

Automatic Generation Control for Distributed Multi-Region Interconnected Power System with Function Approximation

On Training Effective Reinforcement Learning Agents for Real-time Power Grid Operation and Control

Soft actor-critic –based multi-objective optimized energy conversion and management strategy for integrated energy systems with renewable energy

Real-time Optimisation of Short-Term Frequency Stability Controls for a Power System with Renewables and Multi-Infeed HVDCs

A Cooperative Multi-Agent Q-Learning Control Framework for Real-Time Energy Management in Energy Communities

Real-Time Optimal Energy and Reserve Management of Electric Vehicle Fast Charging Station: Hierarchical Game Approach

Stackelberg Game Based Energy and Reserve Management for a Fast Electric Vehicle Charging Station

Distributed Economic Automatic Generation Control: A Game Theoretic Perspective

Grid-area coordinated load frequency control strategy using large-scale multi-agent deep reinforcement learning

Low Carbon Oriented Multi-Stage Energy Management of Integrated Energy Systems Considering Spatio-Temporal Correlation Characteristics and Operational Uncertainties