Abstract:In this article, an environment that represents a novel framework of TEM is designed, all participants send their bids to this framework and receive their profit from it. Also, new state‐action spaces are designed for sellers and buyers so that they can apply the Soft Actor‐Critic (SAC) algorithm to converge to the best policy. A brief of this algorithm, which is for continuous state‐action space, is described. This algorithm is implemented for both single agent (a seller and a buyer) and Multi‐Agent cases. Thanks to reinforcement learning (RL), decision‐making is more convenient and more economical in different situations with high uncertainty. In line with the same fact, it is proposed that prosumers can apply RL to earn more profit in the transactive energy market (TEM). In this article, an environment that represents a novel framework of TEM is designed, where all participants send their bids to this framework and receive their profit from it. Also, new state‐action spaces are designed for sellers and buyers so that they can apply the Soft Actor‐Critic (SAC) algorithm to converge to the best policy. A brief of this algorithm, which is for continuous state‐action space, is described. First, this algorithm is implemented for a single agent (a seller and a buyer). Then we consider all players including sellers and buyers who can apply this algorithm as Multi‐Agent. In this situation, there is a comprehensive game between participants that is investigated, and it is analyzed whether the players converge to the Nash equilibrium (NE) in this game. Finally, numerical results for the IEEE 33‐bus distribution power system illustrate the effectiveness of the new framework for TEM, increasing sellers' and buyers' profits by applying SAC with the new state‐action spaces. SAC is implemented as a Multi‐Agent, demonstrating that players converge to a singular or one of the multiple NEs in this game. The results demonstrate that buyers converge to their optimal policies within 80 days, while sellers achieve optimality after 150 days in the games created between all participants.

A Critical Study on Multi-agent System Based on Reinforcement Learning Theory and its Application in Research of Electricity Market Simulation

Multi-agent simulation-based residential electricity pricing schemes design and user selection decision-making

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Applications of Reinforcement Learning in Deregulated Power Market: A Comprehensive Review

Reinforcement learning in deregulated energy market: A comprehensive review

How to Use Reinforcement Learning to Facilitate Future Electricity Market Design? Part 1: A Paradigmatic Theory

A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids

Study on MAS-Based Electricity Market Operation System

THEORETICAL FUNDAMENTAL AND CONCEPTS OF ELECTRICITY MARKET SIMULATION BASED ON AGENTS' BELIEF LEARNGING

A SURVEY ON AGENT BASED ELECTRICITY MARKET SIMULATION

Study of power market simulation based on distributed multi-agent system

Research on the MAS Architecture of Power Market Simulating System

Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market

Deep Multi-Task Multi-Agent Reinforcement Learning Based Joint Bidding and Pricing Strategy of Price-Maker Load Serving Entity

Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior

Multi-Agent Reinforcement Learning for Power System Operation and Control

A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling

Machine learning applications for electricity market agent-based models: A systematic literature review

Multi‐agent reinforcement learning in a new transactive energy mechanism