Abstract:The increasing penetration of small-scale distributed energy resources (DER) has the potential to support cost-efficient energy balancing in emerging electricity systems, but is also fundamentally affecting the conventional operation paradigm of the latter. In this context, innovative market mechanisms need to be devised to better coordinate and provide incentives for DER to utilize their flexibility. Peer-to-Peer (P2P) energy trading has emerged as an alternative approach to facilitate direct trading between consumers and prosumers interacting in an energy collective and fosters more efficient local demand–supply balancing. While previous research has primarily focused on the technical and economic benefits of P2P trading, little effort has been made towards the incorporation of prosumers’ heterogeneous characteristics in the P2P trading problem. Here, we address this research gap by classifying the participating prosumers into multiple clusters with regard to their portfolio of DER, and analyzing their trading decisions in a simulated P2P trading platform. The latter employs the mid-market rate (MMR) local pricing mechanism to enable energy trading among prosumers and penalizes the contribution to the system demand peak of each prosumer. We formulate the P2P trading problem as a multi-agent coordination problem and propose a novel multi-agent deep reinforcement learning (MADRL) method to address it. The proposed method is founded on the combination of the multi-agent deep deterministic policy gradient (MADDPG) algorithm and the technique of parameter sharing (PS), which not only enables accelerating the training speed by sharing experiences and learned policies between all agents in each cluster, but also sustains the policies’ diversity between multiple clusters. To address the non-stationarity and computational complexity of MADRL as well as persevering the privacy of prosumers, the P2P trading platform acts as a trusted third party which augments the market collective trading information to help training of prosumer agents. Experiments with a large-scale real-world data-set involving 300 residential households demonstrate that the proposed MADRL method exhibits a strong generalization capability in the test data-set and outperforms the state-of-the-art MADRL methods with regard to the system operation cost, demand peak as well as computational time.

Distributed Energy Trading and Scheduling Among Microgrids via Multiagent Reinforcement Learning

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning

Multiagent Reinforcement Learning with Learning Automata for Microgrid Energy Management and Decision Optimization

Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling

Multi-energy Management of Interconnected Multi-microgrid System Using Multi-agent Deep Reinforcement Learning

Peer-to-Peer Energy Trading and Energy Conversion in Interconnected Multi-Energy Microgrids Using Multi-Agent Deep Reinforcement Learning

Peer-to-peer energy trading with energy trading consistency in interconnected multi-energy microgrids: A multi-agent deep reinforcement learning approach

Joint Energy and Carbon Trading for Multi-Microgrid System Based on Multi-Agent Deep Reinforcement Learning

Multi-Microgrid Collaborative Optimization Scheduling Using an Improved Multi-Agent Soft Actor-Critic Algorithm

Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization

Multistep Multiagent Reinforcement Learning for Optimal Energy Schedule Strategy of Charging Stations in Smart Grid.

Multi-agent Deep Reinforcement Learning for Microgrid Energy Scheduling

Microgrid Optimization Strategy for Charging and Swapping Power Stations with New Energy Based on Multi-Agent Reinforcement Learning

Multi-microgrid Energy Management Based on Finite Horizon Twin Delayed Deep Deterministic Policy Gradient

Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids

AI-Powered Microgrid Networks: Multi-Agent Deep Reinforcement Learning for Optimized Energy Trading in Interconnected Systems

Deep Reinforcement Learning and Blockchain for Peer-to-Peer Energy Trading among Microgrids

Scalable Coordinated Management of Peer-to-peer Energy Trading: A Multi-Cluster Deep Reinforcement Learning Approach