Abstract:With the participation of renewable energy sources (RESs) in the electricity market, the inherent uncertainty brings new challenges for strategic bidding and market clearing in the electricity market. In this paper, the market structure is formulated as a non-cooperative game among power suppliers, consumers, and the market operator. A reinforcement-probability Bayesian approach is proposed to obtain the optimal bidding strategy of RESs and achieve maximal social welfare at the market clearing, considering uncertain cost types of RESs and incomplete market information. First, a reinforcement-probability model is presented to obtain the optimal agent-type-strategy combination considering the uncertainty of cost type selection. A agent-type-probability algorithm is proposed for the model to update the mean of probability density function of cost types according to the expected profit. Second, an incremental-cost-based distributed algorithm is proposed to solve the market clearing problem, which accelerates the convergence rate without violating the system constraints via feedback from the scheduled power. Finally, a Bayesian probability model is proposed for RESs to learn the hidden information of the market and transform the incomplete information electricity market into a complete one. A Bayesian Nash equilibrium is reached in the model. To coordinate the information among market participants, the output of the Bayesian probability model is calculated by the Hysteretic Q-learning algorithm and used as the reward to update the agent-type-probability in the reinforcement-probability model. An illustrative example is presented to demonstrate the effectiveness of the proposed approach, which shows that the proposed approach has a better convergence performance and has a 2–6 times faster learning speed compared with the policy gradient and deep Q-network.

Strategic Bidding of the Electricity Producers Based on the Reinforcement Learning

Modeling the Strategic Bidding of the Producers in Competitive Electricity Markets with the Watkins' Q (lambda) Reinforcement Learning

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

A Reinforcement Learning Method for Power Suppliers' Strategic Bidding with Insufficient Information

Bidding Strategy of Two-Layer Optimization Model for Electricity Market Considering Renewable Energy Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Price-Quantity-Pair Bidding in Electricity Markets

Reinforcement Learning Model For Optimizing Generation Bidding Strategy With Risk Management

Bidding Strategic of Virtual Power Plant Based on End-to-End Deep Reinforcement Learning

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

Research and application of the Q-learning for wholesale power markets

Realization of Decision-making Module in Agent-based Simulation of Power Markets

A Multiagent Competitive Bidding Strategy in a Pool-Based Electricity Market With Price-Maker Participants of WPPs and EV Aggregators

Selective Learning for Strategic Bidding in Uniform Pricing Electricity Spot Market

Deep Multi-Task Multi-Agent Reinforcement Learning Based Joint Bidding and Pricing Strategy of Price-Maker Load Serving Entity

Multi-Agent Deep Reinforcement Learning for Simulating Centralized Double-Sided Auction Electricity Market

A reinforcement-probability Bayesian approach for strategic bidding and market clearing for renewable energy sources with uncertainty

Optimal Bidding Strategy of Generation Companies Based on Heuristic Dynamic Programming

Bi-level stochastic real-time pricing model in multi-energy generation system: A reinforcement learning approach

Modeling and Simulation of Bidding Activities of Power Generation Companies by Multi-agent