Abstract:In this paper, we address the problem of jointly determining the energy bid submitted to the wholesale electricity market (WEM) and the energy price charged in the retailed electricity market (REM) for a load serving entity (LSE). The joint bidding and pricing problem is formulated as a Markov decision process (MDP) with continuous state and action spaces, in which the energy bid and the energy price are two actions that share a common objective. We apply the deep deterministic policy gradient (DDPG) algorithm to solve this MDP for the optimal bidding and pricing policies. Yet, the DDPG algorithm typically requires a significant number of state transition samples, which is costly in this application. To this end, we apply neural networks to learn dynamical bid and price response functions from historical data to model the WEM and the collective behavior of the EUCs, respectively. These response functions explicitly capture the inter-temporal correlations of the WEM clearing results and the EUC responses, and can be utilized to generate state transition samples without any cost. More importantly, the response functions also inform the choice of states in the MDP formulation. Numerical simulations illustrated the effectiveness of the proposed methodology. IEEE Transactions on smart grids This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c © Mitsubishi Electric Research Laboratories, Inc., 2020 201 Broadway, Cambridge, Massachusetts 02139

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity /Author=Xu, Hanchen; Sun, Hongbo; Nikovski, Daniel N.; Kitamura, Shoichi; Mori, Kazuyuki; Hashimoto, Hiroyuki /CreationDate=January 8, 2020 /Subject=Artificial Intelligence, Data Analytics, Electric Systems, Machine Lea

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Integrated Demand Response for a Load Serving Entity in Multi-Energy Market Considering Network Constraints

Deep Reinforcement Learning-Based Trading Strategy for Load Aggregators on Price-Responsive Demand

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

A data‐driven method for microgrid bidding optimization in electricity market

Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets

Bidding Strategic of Virtual Power Plant Based on End-to-End Deep Reinforcement Learning

Reinforcement Learning Based Bidding Framework with High-dimensional Bids in Power Markets

Data-driven online interactive bidding strategy for demand response

High-dimensional Bid Learning for Energy Storage Bidding in Energy Markets

Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

ReLeDP: Reinforcement-Learning-Assisted Dynamic Pricing for Wireless Smart Grid

Dynamic Matching Markets in Power Grid: Concepts and Solution using Deep Reinforcement Learning

Reinforcement Learning for Bidding Strategy Optimization in Day-Ahead Energy Market

Dynamic Demand-Aware Power Grid Intelligent Pricing Algorithm Based on Deep Reinforcement Learning

Machine Learning-Driven Virtual Bidding with Electricity Market Efficiency Analysis

A Hierarchical Deep Reinforcement Learning-Based Community Energy Trading Scheme for a Neighborhood of Smart Households

Price-Based Residential Demand Response Management in Smart Grids: A Reinforcement Learning-Based Approach