Abstract:Residential and commercial buildings are responsible for 35% of the EU energy-related greenhouse gas (GHG) emissions. Reducing their emissions is crucial for meeting the challenging EU objective of the agenda for becoming a net-zero continent by 2050. The diffusion and integration of distributed renewable energy sources (RESs) and energy storage systems (ESSs), as well as the creation of energy communities (ECs), have proven to be crucial aspects in reducing GHG emissions. In this context, this article proposes a multi-agent AI-based control framework to solve the EC's energy management problem in the presence of distributed RESs and ESSs as well as considering a shared ESS. The objectives of the proposed control framework are to satisfy the EC members' load demand to maximize self-consumption and to manage ESSs charging and discharging processes, to enforce cooperative behavior among the EC members by adopting fair and personalized strategies and to maximize EC members' profits. The proposed control procedure is based on three sequential stages, each solved by a dedicated local RL agent exploiting the Q-Learning algorithm. To reduce the computational complexity of the proposed approach, specifically defined state aggregation criteria were defined to map the RL agents' continuous state spaces into discrete state spaces of limited dimensions. During the training phase, the EC members' profiles and the ESSs' and RESs' characteristics were randomly changed to allow the RL agents to learn the correct policy to follow in any given scenario. Simulations proved the effectiveness of the proposed approach for different costumers' load demand profiles and different EC configurations. Indeed, the trained RL agents proved to be able to satisfy the EC members' load demands to maximize self-consumption, to correctly use the distributed and shared ESSs, to charge them according to respective personalized criteria and to sell the energy surplus, prioritizing sales to the EC. The proposed control framework also proved to be a useful tool for understanding EC performance in different configurations and, thus, for properly dimensioning the EC elements.

Evaluating Inter-Operator Cooperation Scenarios to Save Radio Access Network Energy

On energy-optimal cooperation strategy in cognitive radio networks

Balancing Performance and Cost for Two-Hop Cooperative Communications: Stackelberg Game and Distributed Multi-Agent Reinforcement Learning

DRL-based Energy-Efficient Baseband Function Deployments for Service-Oriented Open RAN

Energy Consumption Optimization in Radio Access Networks (ECO-RAN)

Collaborative Intelligent Resource Trading for RAN Slicing: A Federated Policy Distillation Approach

Cooperative Resource Trading for Network Slicing in Industrial IoT: A Multi-Agent DRL Approach

Collaborative Multi-BS Power Management for Dense Radio Access Network using Deep Reinforcement Learning

Collaborative Task Offloading and Resource Allocation in Small-Cell MEC: A Multi-Agent PPO-Based Scheme

Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell Massive MIMO Systems

Multi-Agent Deep Reinforcement Learning for Computation Offloading and Interference Coordination in Small Cell Networks

Learning to run a power network with trust

Design and Evaluation of Deep Reinforcement Learning for Energy Saving in Open RAN

A Cooperative Multi-Agent Q-Learning Control Framework for Real-Time Energy Management in Energy Communities

Energy Saving Through a Learning Framework in Greener Cellular Radio Access Networks.

Multi-Agent-Deep-Reinforcement-Learning-Enabled Offloading Scheme for Energy Minimization in Vehicle-to-Everything Communication Systems

Cooperative Relaying Strategies for Smart Grid Communications: Bargaining Models and Solutions

AoI-Oriented Resource Allocation for NOMA-Based Wireless Powered Cognitive Radio Networks Based on Multi-Agent Deep Reinforcement Learning

Energy-Efficient Joint Resource Allocation With Reconfigurable Intelligent Surfaces in Symbiotic Radio Networks

Multi-Agent Deep Reinforcement Learning for Energy Efficient Multi-Hop STAR-RIS-Assisted Transmissions

Reinforcement-Learning-Based Robust Resource Management for Multi-Radio Systems