Abstract:To reduce global greenhouse gas emissions, the world must find intelligent solutions to maximise the utilisation of carbon-free renewable energy sources. In this paper, multi-agent reinforcement learning is used to control a microgrid in a mixed cooperative and competitive setting. The agents observe fluctuating energy demand, dynamic wholesale energy prices, and intermittent renewable energy sources to control a hybrid energy storage system to maximise the utilisation of the renewables to reduce the energy costs of the grid. In addition, an aggregator agent trades with external microgrids competing against one another and the aggregator to reduce their own energy bills. For this, the algorithm deep deterministic policy gradients (DDPG) and multi-agent DDPG (MADDPG) are used to compare the use of a single global controller versus multiple distributed agents, along with the single and multi-agent variants of distributional DDPG (D3PG) and twin delayed DDPG (TD3). The research found it is significantly more profitable for the primary microgrid to sell energy on its own terms rather than selling back to the utility grid, and is also beneficial for the external microgrids as they also reduce their own energy bills. The methods that produced the greatest profits were the multi-agent approaches where each agent has its own reward function based on the principle of marginal contribution from game theory. The multi-agent approaches were better able to evaluate their performance controlling individual components of the environment which allowed them to develop their own unique policies for the different types of energy storage system.

Demand-Side Frequency Control in Power Network Based on Multi-Agent Deep Reinforcement Learning

Model-Free Load Frequency Control Based on Multi-Agent Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

A Multi-Agent Deep Reinforcement Learning-Based “octopus” Cooperative Load Frequency Control for an Interconnected Grid with Various Renewable Units

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Coordinated load frequency control of multi-area integrated energy system using multi-agent deep reinforcement learning

Multi-Agent Reinforcement Learning for Smart Community Energy Management

Data-Driven Load Frequency Control Based on Multi-Agent Reinforcement Learning With Attention Mechanism

Multi-agent Deep Reinforcement Learning Strategy for Distributed Energy

Optimal operation of regional integrated energy system based on multi-agent deep deterministic policy gradient algorithm

Deep Reinforcement Learning Based Optimal Scheduling of Active Distribution System Considering Distributed Generation, Energy Storage and Flexible Load

A Multiagent Reinforcement Learning Approach for Wind Farm Frequency Control

Grid-area coordinated load frequency control strategy using large-scale multi-agent deep reinforcement learning

Data‐driven cooperative load frequency control method for microgrids using effective exploration‐distributed multi‐agent deep reinforcement learning

Deep reinforcement learning for adaptive frequency control of island microgrid considering control performance and economy

Optimal demand response based dynamic pricing strategy via Multi-Agent Federated Twin Delayed Deep Deterministic policy gradient algorithm

Distributed deep reinforcement learning for integrated generation‐control and power‐dispatch of interconnected power grid with various renewable units

Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning

Multiagent deep meta reinforcement learning for sea computing-based energy management of interconnected grids considering renewable energy sources in sustainable cities

Multi-timescale voltage control for distribution system based on multi-agent deep reinforcement learning

PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control