Abstract:The bidding behaviors of the energy storage systems (ESS) are complicated due to time coupling and market coupling limited by their capacity states. The existing research is mainly based on optimization models and reinforcement learning (RL) models, which are idealized with analytical objective functions, rational decisions, and virtual historical data. This leads to mismatches between theoretical results and actual bidding behaviors, and the obtained results cannot be generalized to a real-world market. In recent years, the disclosure of market data has enabled data-driven analysis, while the complexity of actual data makes the analysis process challenging and calls for more suitable methods. On the basis of our investigation of ESS bidding behaviors and market data, we propose a novel inverse RL (IRL)-based framework to identify the bidding decision objective function of ESS in coupled multi-market through their historical bidding records and operation status. The identification results can be used as the model parameters to help RL-based simulation models available for a real-world market. Based on the proposed framework, we conduct an empirical analysis on the real-world data of a battery ESS in the Australian electricity market. The results show how the ESS balances its objective among different markets and maintains its state of charge (SOC) to gain more profits. This framework can help us model ESS behaviors by relying on historical data to better understand the ESS bidding decision-making mechanism, which will facilitate market equilibrium analysis and mechanism design.

Bidding Strategy Evolution Analysis Based on Multi-Task Inverse Reinforcement Learning

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Optimal Pricing Strategy of Power-Retailing Companies Considering Demand Response in Block Bidding Markets

Deep Inverse Reinforcement Learning for Objective Function Identification in Bidding Models

Bi-Level Programming-Based Optimal Strategy to LSEs with Demand Response Bids

Multi-Market Bidding Behavior Analysis of Energy Storage System Based on Inverse Reinforcement Learning

Multi-Market Bidding Strategy of Power Suppliers

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

Bidding Strategic of Virtual Power Plant Based on End-to-End Deep Reinforcement Learning

Bidding Strategy of Two-Layer Optimization Model for Electricity Market Considering Renewable Energy Based on Deep Reinforcement Learning

A Data-Driven Pattern Extraction Method for Analyzing Bidding Behaviors in Power Markets

Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning

BIDDING STRATEGY FOR ELECTRIC POWER SUPPLIERS IN THE DAILY ENERGY MARKET

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

Strategic Bidding of the Electricity Producers Based on the Reinforcement Learning

Preliminary Study on Bidding Strategy for Electricity Market

An Ordinal Optimization Based Bidding Strategy for Electric Power Suppliers in the Daily Energy Market

Modeling the Strategic Bidding of the Producers in Competitive Electricity Markets with the Watkins' Q (lambda) Reinforcement Learning

Reinforcement Learning Model For Optimizing Generation Bidding Strategy With Risk Management

Forecasting individual bids in real electricity markets through machine learning framework