Abstract:The increasing penetration of small-scale distributed energy resources (DER) has the potential to support cost-efficient energy balancing in emerging electricity systems, but is also fundamentally affecting the conventional operation paradigm of the latter. In this context, innovative market mechanisms need to be devised to better coordinate and provide incentives for DER to utilize their flexibility. Peer-to-Peer (P2P) energy trading has emerged as an alternative approach to facilitate direct trading between consumers and prosumers interacting in an energy collective and fosters more efficient local demand–supply balancing. While previous research has primarily focused on the technical and economic benefits of P2P trading, little effort has been made towards the incorporation of prosumers’ heterogeneous characteristics in the P2P trading problem. Here, we address this research gap by classifying the participating prosumers into multiple clusters with regard to their portfolio of DER, and analyzing their trading decisions in a simulated P2P trading platform. The latter employs the mid-market rate (MMR) local pricing mechanism to enable energy trading among prosumers and penalizes the contribution to the system demand peak of each prosumer. We formulate the P2P trading problem as a multi-agent coordination problem and propose a novel multi-agent deep reinforcement learning (MADRL) method to address it. The proposed method is founded on the combination of the multi-agent deep deterministic policy gradient (MADDPG) algorithm and the technique of parameter sharing (PS), which not only enables accelerating the training speed by sharing experiences and learned policies between all agents in each cluster, but also sustains the policies’ diversity between multiple clusters. To address the non-stationarity and computational complexity of MADRL as well as persevering the privacy of prosumers, the P2P trading platform acts as a trusted third party which augments the market collective trading information to help training of prosumer agents. Experiments with a large-scale real-world data-set involving 300 residential households demonstrate that the proposed MADRL method exhibits a strong generalization capability in the test data-set and outperforms the state-of-the-art MADRL methods with regard to the system operation cost, demand peak as well as computational time.

A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids

Distributed Energy Management and Demand Response in Smart Grids: A Multi-Agent Deep Reinforcement Learning Framework

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

AI-Powered Microgrid Networks: Multi-Agent Deep Reinforcement Learning for Optimized Energy Trading in Interconnected Systems

Enhancing Distributed Energy Markets in Smart Grids Through Game Theory and Reinforcement Learning

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

A Hierarchical Deep Reinforcement Learning-Based Community Energy Trading Scheme for a Neighborhood of Smart Households

Scalable Coordinated Management of Peer-to-peer Energy Trading: A Multi-Cluster Deep Reinforcement Learning Approach

Multi-agent deep reinforcement learning-based optimal energy management for grid-connected multiple energy carrier microgrids

Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning

Deep Reinforcement Learning for Price-Quantity-Pair Bidding in Electricity Markets

Multi-Agent Deep Reinforcement Learning for Blockchain-Based Energy Trading in Decentralized Electric Vehicle Charger-Sharing Networks

Energy Trading in Smart Grid: A Deep Reinforcement Learning-based Approach

Reinforcement Learning for Fair and Efficient Charging Coordination for Smart Grid

A Deep Reinforcement Learning-Based Bidding Strategy for Participants in a Peer-to-peer Energy Trading Scenario

Decentralized Coordination of Distributed Energy Resources through Local Energy Markets and Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning for Simulating Centralized Double-Sided Auction Electricity Market

Peer-to-peer energy trading with energy trading consistency in interconnected multi-energy microgrids: A multi-agent deep reinforcement learning approach

Reinforcement Learning in Energy Trading Game among Smart Microgrids.

Mean-Field Multi-Agent Reinforcement Learning for Peer-to-Peer Multi-Energy Trading