Abstract:To reduce global greenhouse gas emissions, the world must find intelligent solutions to maximise the utilisation of carbon-free renewable energy sources. In this paper, multi-agent reinforcement learning is used to control a microgrid in a mixed cooperative and competitive setting. The agents observe fluctuating energy demand, dynamic wholesale energy prices, and intermittent renewable energy sources to control a hybrid energy storage system to maximise the utilisation of the renewables to reduce the energy costs of the grid. In addition, an aggregator agent trades with external microgrids competing against one another and the aggregator to reduce their own energy bills. For this, the algorithm deep deterministic policy gradients (DDPG) and multi-agent DDPG (MADDPG) are used to compare the use of a single global controller versus multiple distributed agents, along with the single and multi-agent variants of distributional DDPG (D3PG) and twin delayed DDPG (TD3). The research found it is significantly more profitable for the primary microgrid to sell energy on its own terms rather than selling back to the utility grid, and is also beneficial for the external microgrids as they also reduce their own energy bills. The methods that produced the greatest profits were the multi-agent approaches where each agent has its own reward function based on the principle of marginal contribution from game theory. The multi-agent approaches were better able to evaluate their performance controlling individual components of the environment which allowed them to develop their own unique policies for the different types of energy storage system.

Peer-to-peer energy trading with energy trading consistency in interconnected multi-energy microgrids: A multi-agent deep reinforcement learning approach

Peer-to-Peer Energy Trading and Energy Conversion in Interconnected Multi-Energy Microgrids Using Multi-Agent Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Peer-to-peer energy trading optimization in energy communities using multi-agent deep reinforcement learning

Multi-agent deep deterministic policy gradient algorithm for peer-to-peer energy trading considering distribution network constraints

Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning

Distributed Energy Trading and Scheduling Among Microgrids via Multiagent Reinforcement Learning

Mean-Field Multi-Agent Reinforcement Learning for Peer-to-Peer Multi-Energy Trading

Peer-to-Peer Energy Trading of Solar and Energy Storage: A Networked Multiagent Reinforcement Learning Approach

Joint Energy and Carbon Trading for Multi-Microgrid System Based on Multi-Agent Deep Reinforcement Learning

Peer-to-Peer Trading for Energy-Saving Based on Reinforcement Learning

A Scalable Privacy-Preserving Multi-Agent Deep Reinforcement Learning Approach for Large-Scale Peer-to-Peer Transactive Energy Trading

Peer-to-Peer Multienergy and Communication Resource Trading for Interconnected Microgrids

A microgrid power trading framework based on blockchain and deep reinforcement learning

Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

2A Reliable Energy Trading Strategy in Intelligent Microgrids Using Deep Reinforcement Learning

Microgrid Optimization Strategy for Charging and Swapping Power Stations with New Energy Based on Multi-Agent Reinforcement Learning

Multi-Microgrid Based Energy Trading Framework from Game Theoretic Perspective

Peer-to-peer multi-energy distributed trading for interconnected microgrids: A general Nash bargaining approach

Peer-to-Peer Energy Transactions for Prosumers Based on Improved Deep Deterministic Policy Gradient Algorithm