Abstract:Previously works on analysing imperfect electricity markets have employed conventional game-theoretic approaches. However, such approaches necessitate that each strategic market player has full knowledge of the operating parameters and the strategies of its rivals as well as the computational algorithm of the market clearing process. This unrealistic assumption, along with the modeling and computational complexities, renders such approaches less applicable for conducting practical multi-period and multi-spatial equilibrium analysis. This paper proposes a novel multi-agent deep reinforcement learning (MA-DRL) based methodology, combining multi-agent intelligence, the deep policy gradient (DPG) method, and an innovative long short term memory (LSTM) based representation network for optimizing the offering strategies of multiple self-interested generation companies (GENCOs) as well as exploring the market outcome stemming from their interactions. The proposed approach is tailored to align with the nature of the examined problem by posing it, for the first time, in multi-dimensional continuous state and action spaces, enabling GENCOs to receive accurate feedback regarding the impact of their offering strategies on the market clearing outcome, and devise more profitable bidding decisions by exploiting the entire action domain, and thereby facilitates more accurate equilibrium analysis. The proposed LSTM-based representation network extracts discriminative features which further improves the learning performance and thus promises more profitable offerings strategies for each GENCO. Case studies demonstrate that the proposed method i) achieves a significantly higher profit than state-of-the-art RL methods for a single GENCO's optimal offering strategy problem and ii) outperforms the state-of-the-art equilibrium programming models in efficiently identifying an imperfect market equilibrium with/without network congestion. Quantitative economic analysis is carried out on the obtained equilibrium.

An Agent-Based Approach Equipped with Game Theory: Strategic Collaboration among Learning Agents During a Dynamic Market Change in the California Electricity Crisis

An Agent Based System for California Electricity Market: A Perspective of Myopic Machine Learning

An Agent-Based Decision Support System for Wholesale Electricity Market

A Use of Multi-Agent Intelligent Simulator to Measure the Dynamics of US Wholesale Power Trade: A Case Study of the California Electricity Crisis

Agent-Based Approach to Handle Business Complexity in U.S. Wholesale Power Trading

Why Did California Electricity Crisis Occur? A View of Multi-Agent Intelligent Simulator

An Electric Power Trading System: Network-based Framework and Simulator with Learning Capabilities

An Agent-Based Approach with Collaboration among Agents: Estimation of Wholesale Electricity Price on PJM and Artificial Data Generated by a Mean Reverting Model

A Use of Multi-Agent Intelligent Simulator to Examine California Electricity Crisis

Application of Intelligent Agent-Based Simulation to Electricity Market

A SURVEY ON AGENT BASED ELECTRICITY MARKET SIMULATION

A Wholesale Power Trading Simulator with Learning Capabilities

Intelligent Agent Technology: an Application to US Wholesale Power Trading

Agent-Based Intelligence System for Us Wholesale Power Transaction

Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review

Game theoretical multi-agent modelling of coalition formation for multilateral trades

Many learning agents interacting with an agent-based market model

A simple learning agent interacting with an agent-based market model

Strategic Integration of Distributed Generation in a Deregulated Power Market: An Agent-Based Approach

Multi-Period and Multi-Spatial Equilibrium Analysis in Imperfect Electricity Markets: A Novel Multi-Agent Deep Reinforcement Learning Approach.

Dynamic Matching Markets in Power Grid: Concepts and Solution using Deep Reinforcement Learning