Abstract:In this paper, we address the problem of jointly determining the energy bid submitted to the wholesale electricity market (WEM) and the energy price charged in the retailed electricity market (REM) for a load serving entity (LSE). The joint bidding and pricing problem is formulated as a Markov decision process (MDP) with continuous state and action spaces, in which the energy bid and the energy price are two actions that share a common objective. We apply the deep deterministic policy gradient (DDPG) algorithm to solve this MDP for the optimal bidding and pricing policies. Yet, the DDPG algorithm typically requires a significant number of state transition samples, which is costly in this application. To this end, we apply neural networks to learn dynamical bid and price response functions from historical data to model the WEM and the collective behavior of the EUCs, respectively. These response functions explicitly capture the inter-temporal correlations of the WEM clearing results and the EUC responses, and can be utilized to generate state transition samples without any cost. More importantly, the response functions also inform the choice of states in the MDP formulation. Numerical simulations illustrated the effectiveness of the proposed methodology. IEEE Transactions on smart grids This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c © Mitsubishi Electric Research Laboratories, Inc., 2020 201 Broadway, Cambridge, Massachusetts 02139

Deep Differentiable Reinforcement Learning for Generic Energy Storage Day-Ahead Bidding with Transformer Neural Networks

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets

High-dimensional Bid Learning for Energy Storage Bidding in Energy Markets

Bidding Strategic of Virtual Power Plant Based on End-to-End Deep Reinforcement Learning

Deep Reinforcement Learning-Based Trading Strategy for Load Aggregators on Price-Responsive Demand

Energy Storage Arbitrage in Two-settlement Markets: A Transformer-Based Approach

Reinforcement Learning Based Bidding Framework with High-dimensional Bids in Power Markets

Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity

A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding

Multi-Agent Deep Reinforcement Learning for Blockchain-Based Energy Trading in Decentralized Electric Vehicle Charger-Sharing Networks

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

Deep Learning-Based Electricity Price Forecast for Virtual Bidding in Wholesale Electricity Market

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

Transferable Energy Storage Bidder

A Hybrid Data-Driven Method for Low-Carbon Economic Energy Management Strategy in Electricity-Gas Coupled Energy Systems Based on Transformer Network and Deep Reinforcement Learning

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity /Author=Xu, Hanchen; Sun, Hongbo; Nikovski, Daniel N.; Kitamura, Shoichi; Mori, Kazuyuki; Hashimoto, Hiroyuki /CreationDate=January 8, 2020 /Subject=Artificial Intelligence, Data Analytics, Electric Systems, Machine Lea

Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information