Abstract:In this paper, we address the problem of jointly determining the energy bid submitted to the wholesale electricity market (WEM) and the energy price charged in the retailed electricity market (REM) for a load serving entity (LSE). The joint bidding and pricing problem is formulated as a Markov decision process (MDP) with continuous state and action spaces, in which the energy bid and the energy price are two actions that share a common objective. We apply the deep deterministic policy gradient (DDPG) algorithm to solve this MDP for the optimal bidding and pricing policies. Yet, the DDPG algorithm typically requires a significant number of state transition samples, which is costly in this application. To this end, we apply neural networks to learn dynamical bid and price response functions from historical data to model the WEM and the collective behavior of the EUCs, respectively. These response functions explicitly capture the inter-temporal correlations of the WEM clearing results and the EUC responses, and can be utilized to generate state transition samples without any cost. More importantly, the response functions also inform the choice of states in the MDP formulation. Numerical simulations illustrated the effectiveness of the proposed methodology. IEEE Transactions on smart grids This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c © Mitsubishi Electric Research Laboratories, Inc., 2020 201 Broadway, Cambridge, Massachusetts 02139

Energy Procurement and Retail Pricing for Electricity Retailers via Deep Reinforcement Learning with Long Short-term Memory

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Influence of Discrete and Continuous Action Spaces on Deep Reinforcement Learning-Based Pricing Strategy Optimization for Electricity Retailers

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

Deep Reinforcement Learning-Based Trading Strategy for Load Aggregators on Price-Responsive Demand

A Deep Reinforcement Learning-Based Bidding Strategy for Participants in a Peer-to-peer Energy Trading Scenario

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity /Author=Xu, Hanchen; Sun, Hongbo; Nikovski, Daniel N.; Kitamura, Shoichi; Mori, Kazuyuki; Hashimoto, Hiroyuki /CreationDate=January 8, 2020 /Subject=Artificial Intelligence, Data Analytics, Electric Systems, Machine Lea

Integrated Demand Response in Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

Deep Differentiable Reinforcement Learning for Generic Energy Storage Day-Ahead Bidding with Transformer Neural Networks

Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets

Multi-Period and Multi-Spatial Equilibrium Analysis in Imperfect Electricity Markets: A Novel Multi-Agent Deep Reinforcement Learning Approach.

Dynamic Pricing Strategy in Electricity Trading Market Based on Reinforcement Learning

A Hierarchical Deep Reinforcement Learning-Based Community Energy Trading Scheme for a Neighborhood of Smart Households

Deep Multi-Task Multi-Agent Reinforcement Learning Based Joint Bidding and Pricing Strategy of Price-Maker Load Serving Entity

A Reinforcement Learning-Based Decision System for Electricity Pricing Plan Selection by Smart Grid End Users.

Dynamic Pricing on E-commerce Platform with Deep Reinforcement Learning: A Field Experiment

Deep Reinforcement Learning Based Optimal Energy Management of Multi-energy Microgrids with Uncertainties

Arbitrage of Energy Storage in Electricity Markets with Deep Reinforcement Learning

Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting