Abstract:Previously works on analysing imperfect electricity markets have employed conventional game-theoretic approaches. However, such approaches necessitate that each strategic market player has full knowledge of the operating parameters and the strategies of its rivals as well as the computational algorithm of the market clearing process. This unrealistic assumption, along with the modeling and computational complexities, renders such approaches less applicable for conducting practical multi-period and multi-spatial equilibrium analysis. This paper proposes a novel multi-agent deep reinforcement learning (MA-DRL) based methodology, combining multi-agent intelligence, the deep policy gradient (DPG) method, and an innovative long short term memory (LSTM) based representation network for optimizing the offering strategies of multiple self-interested generation companies (GENCOs) as well as exploring the market outcome stemming from their interactions. The proposed approach is tailored to align with the nature of the examined problem by posing it, for the first time, in multi-dimensional continuous state and action spaces, enabling GENCOs to receive accurate feedback regarding the impact of their offering strategies on the market clearing outcome, and devise more profitable bidding decisions by exploiting the entire action domain, and thereby facilitates more accurate equilibrium analysis. The proposed LSTM-based representation network extracts discriminative features which further improves the learning performance and thus promises more profitable offerings strategies for each GENCO. Case studies demonstrate that the proposed method i) achieves a significantly higher profit than state-of-the-art RL methods for a single GENCO's optimal offering strategy problem and ii) outperforms the state-of-the-art equilibrium programming models in efficiently identifying an imperfect market equilibrium with/without network congestion. Quantitative economic analysis is carried out on the obtained equilibrium.

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

A Deep Reinforcement Learning-Based Bidding Strategy for Participants in a Peer-to-peer Energy Trading Scenario

Multi-Agent Deep Reinforcement Learning for Simulating Centralized Double-Sided Auction Electricity Market

Applications of Reinforcement Learning in Deregulated Power Market: A Comprehensive Review

Reinforcement Learning Based Bidding Framework with High-dimensional Bids in Power Markets

Dynamic Pricing Strategy in Electricity Trading Market Based on Reinforcement Learning

Reinforcement learning in deregulated energy market: A comprehensive review

Reinforcement Learning for Bidding Strategy Optimization in Day-Ahead Energy Market

Deep Reinforcement Learning for Price-Quantity-Pair Bidding in Electricity Markets

A Deep Q Network Approach for Optimizing Offering Strategies in Electricity Markets

A Deep Reinforcement Learning Method for Pricing Electric Vehicles with Discrete Charging Levels

Multi-Period and Multi-Spatial Equilibrium Analysis in Imperfect Electricity Markets: A Novel Multi-Agent Deep Reinforcement Learning Approach.

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity

Deep Reinforcement Learning-Based Trading Strategy for Load Aggregators on Price-Responsive Demand

Bidding Strategic of Virtual Power Plant Based on End-to-End Deep Reinforcement Learning

Bidding Strategy of Two-Layer Optimization Model for Electricity Market Considering Renewable Energy Based on Deep Reinforcement Learning

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets

Energy Procurement and Retail Pricing for Electricity Retailers via Deep Reinforcement Learning with Long Short-term Memory