Re: "Management of Marcus Gunn jaw winking synkinesis".

S. Seiff

DOI: https://doi.org/10.1097/01.IOP.0000139528.11188.A6

2004-09-01

Ophthalmic Plastic and Reconstructive Surgery

Abstract:

What problem does this paper attempt to address?

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Yujian Ye,Dawei Qiu,Mingyang Sun,Dimitrios Papadaskalopoulos,Goran Strbac

DOI: https://doi.org/10.1109/tsg.2019.2936142

IF: 10.275

2020-01-01

IEEE Transactions on Smart Grid

Abstract:Bi-level optimization and reinforcement learning (RL) constitute the state-of-the-art frameworks for modeling strategic bidding decisions in deregulated electricity markets. However, the former neglects the market participants' physical non-convex operating characteristics, while conventional RL methods require discretization of state and/or action spaces and thus suffer from the curse of dimensionality. This paper proposes a novel deep reinforcement learning (DRL) based methodology, combining a deep deterministic policy gradient (DDPG) method with a prioritized experience replay (PER) strategy. This approach sets up the problem in multi-dimensional continuous state and action spaces, enabling market participants to receive accurate feedback regarding the impact of their bidding decisions on the market clearing outcome, and devise more profitable bidding decisions by exploiting the entire action domain, also accounting for the effect of non-convex operating characteristics. Case studies demonstrate that the proposed methodology achieves a significantly higher profit than the alternative state-of-the-art methods, and exhibits a more favourable computational performance than benchmark RL methods due to the employment of the PER strategy.
Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

Chuwei Liu,Xuan Rao,Bo Zhao,Derong Liu,Qinglai Wei,Yonghua Wang

DOI: https://doi.org/10.1016/j.epsr.2024.110404

IF: 3.818

2024-05-18

Electric Power Systems Research

Abstract:This paper aims to address the bidding strategy optimization in the real-time multi-participant electricity market with short-term load dynamics. In order to avoid the sub-optimal solution and the dependence on the complete information in traditional mathematical programming methods, an electricity market bidding strategy optimization algorithm based on deep reinforcement learning (DRL) is developed. While conventional reinforcement learning algorithms (e.g., Q-learning and deep Q-learning) are only capable of handling simple problems in discrete state spaces, the proximal policy optimization (PPO) algorithm is implemented in the bidding strategy optimization since it can optimize the bidding strategy in the continuous action and state spaces. In order to substantiate the aforementioned perspective, this paper conducts a two-part experimental study. First, experiments which consider the fixed demand load of market participants show that the developed method can reach the Nash equilibrium just like the bi-level optimization, and higher profits can be achieved by adjusting hyperparameters. Then, complex experiments which consider the time-varying demand load verify the DRL-based electricity market bidding strategy performs better than bi-level optimization-based methods and increases the profits of generators.

engineering, electrical & electronic
Deep Reinforcement Learning Based Pricing Strategy of Aggregators Considering Renewable Energy

Yu-Chieh Chuang,Wei-Yu Chiu

DOI: https://doi.org/10.1109/tetci.2021.3109954

2021-01-01

IEEE Transactions on Emerging Topics in Computational Intelligence

Abstract:With the rapid development of information and communications technology and high penetration of renewable energy, the role of an aggregator in a smart grid has emerged to better coordinate power and cash flows between energy producers and consumers through the adjustment of pricing signals. This study proposes variation indices about the statistics of renewables and a control law for an energy storage system. A deep reinforcement learning based pricing strategy of an aggregator for profit maximization in consideration of the energy balance is developed accordingly. The proposed approach can consider opponents behaviors, variability of renewables, and varying bounds of charging and discharging events in a nonstationary environment, which can be hardly addressed by pricing strategies based on conventional learning algorithms such as Q-learning and deep Q-network. Numerical analysis using real-world data shows that the proposed approach can outperform existing pricing strategies in terms of the learning speed and profit of aggregators.
Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity

Hanchen Xu,Hongbo Sun,Daniel Nikovski,Shoichi Kitamura,Kazuyuki Mori,Hiroyuki Hashimoto

DOI: https://doi.org/10.1109/tsg.2019.2903756

IF: 10.275

2019-11-01

IEEE Transactions on Smart Grid

Abstract:In this paper, we address the problem of jointly determining the energy bid submitted to the wholesale electricity market (WEM) and the energy price charged in the retailed electricity market (REM) for a load serving entity (LSE). The joint bidding and pricing problem is formulated as a Markov decision process (MDP) with continuous state and action spaces in which the energy bid and the energy price are two actions that share a common objective. We apply the deep deterministic policy gradient (DDPG) algorithm to solve this MDP for the optimal bidding and pricing policies. Yet, the DDPG algorithm typically requires a significant number of state transition samples, which are costly in this application. To this end, we apply neural networks to learn dynamical bid and price response functions from historical data to model the WEM and the collective behavior of the end use customers (EUCs), respectively. These response functions explicitly capture the inter-temporal correlations of the WEM clearing results and the EUC responses and can be utilized to generate state transition samples without any cost. More importantly, the response functions also inform the choice of states in the MDP formulation. Numerical simulations illustrated the effectiveness of the proposed methodology.

engineering, electrical & electronic
Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity /Author=Xu, Hanchen; Sun, Hongbo; Nikovski, Daniel N.; Kitamura, Shoichi; Mori, Kazuyuki; Hashimoto, Hiroyuki /CreationDate=January 8, 2020 /Subject=Artificial Intelligence, Data Analytics, Electric Systems, Machine Lea

Hanchen Xu,Hongbo Sun,Daniel Nikovski,Shoichi Kitamura,Hiroyuki Hashimoto

2020-01-01

Abstract:In this paper, we address the problem of jointly determining the energy bid submitted to the wholesale electricity market (WEM) and the energy price charged in the retailed electricity market (REM) for a load serving entity (LSE). The joint bidding and pricing problem is formulated as a Markov decision process (MDP) with continuous state and action spaces, in which the energy bid and the energy price are two actions that share a common objective. We apply the deep deterministic policy gradient (DDPG) algorithm to solve this MDP for the optimal bidding and pricing policies. Yet, the DDPG algorithm typically requires a significant number of state transition samples, which is costly in this application. To this end, we apply neural networks to learn dynamical bid and price response functions from historical data to model the WEM and the collective behavior of the EUCs, respectively. These response functions explicitly capture the inter-temporal correlations of the WEM clearing results and the EUC responses, and can be utilized to generate state transition samples without any cost. More importantly, the response functions also inform the choice of states in the MDP formulation. Numerical simulations illustrated the effectiveness of the proposed methodology. IEEE Transactions on smart grids This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c © Mitsubishi Electric Research Laboratories, Inc., 2020 201 Broadway, Cambridge, Massachusetts 02139
Blockchain-Assisted Secure Energy Trading in Electricity Markets: A Tiny Deep Reinforcement Learning-Based Stackelberg Game Approach

Yong Xiao,Xiaoming Lin,Yiyong Lei,Yanzhang Gu,Jianlin Tang,Fan Zhang,Bin Qian

DOI: https://doi.org/10.3390/electronics13183647

IF: 2.9

2024-09-14

Electronics

Abstract:Electricity markets are intricate systems that facilitate efficient energy exchange within interconnected grids. With the rise of low-carbon transportation driven by environmental policies and tech advancements, energy trading has become crucial. This trend towards Electric Vehicles (EVs) is bolstered by the pivotal role played by EV charging operators in providing essential charging infrastructure and services for widespread EV adoption. This paper introduces a blockchain-assisted secure electricity trading framework between EV charging operators and the electricity market with renewable energy sources. We propose a single-leader, multi-follower Stackelberg game between the electricity market and EV charging operators. In the two-stage Stackelberg game, the electricity market acts as the leader, deciding the price of electric energy. The EV charging aggregator leverages blockchain technology to record and verify energy trading transactions securely. The EV charging operators, acting as followers, then decide their demand for electric energy based on the set price. To find the Stackelberg equilibrium, we employ a Deep Reinforcement Learning (DRL) algorithm that tackles non-stationary challenges through policy, action space, and reward function formulation. To optimize efficiency, we propose the integration of pruning techniques into DRL, referred to as Tiny DRL. Numerical results demonstrate that our proposed schemes outperform traditional approaches.

engineering, electrical & electronic,computer science, information systems,physics, applied
An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Baohui Han,Mingjie Hu,Shilin Lv,Zhejing Bao,Lingxia Lu,Miao Yu

DOI: https://doi.org/10.1109/repe59476.2023.10511299

2023-01-01

Abstract:Integrated demand response represents an effective approach for enhancing energy utilization efficiency while achieving optimal functioning of both supply and demand sides of integrated energy systems. Accounting for the independent response capability of demand users and the inherent uncertainty within the system, an energy management framework is presented based on an enhanced deep reinforcement learning algorithm in this paper. This framework aims to optimize the interaction between the supply and demand sides of integrated energy systems to minimize operational costs. Firstly, an integrated demand response model is introduced, which considers the intricate attributes of multi-energy users and their coupling characteristics with each other. Subsequently, an optimization model is proposed based on an improved deep reinforcement learning algorithm, specifically devised to tackle the challenges posed by system uncertainty. Case simulations show that the proposed method can adapt to the uncertainty of the system and is effective in reducing the energy cost compared to that without the integrated demand response.
A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

Shuai Jia,Zhongxue Gan,Yugeng Xi,Dewei Li,Shibei Xue,Limin Wang

DOI: https://doi.org/10.1007/s11630-020-1308-0

IF: 2.013

2020-08-22

Journal of Thermal Science

Abstract:In this paper, we design a new bidding algorithm by employing a deep reinforcement learning approach. Firms use the proposed algorithm to estimate conjectural variation of the other firms and then employ this variable to generate the optimal bidding strategy so as to pursue maximal profits. With this algorithm, electricity generation firms can improve the accuracy of conjectural variations of competitors by dynamically learning in an electricity market with incomplete information. Electricity market will reach an equilibrium point when electricity firms adopt the proposed bidding algorithm for a repeated game of power trading. The simulation examples illustrate the overall energy efficiency of power network will increase by 9.90% as the market clearing price decreasing when all companies use the algorithm. The simulation examples also show that the power demand elasticity has a positive effect on the convergence of learning process.

engineering, mechanical,thermodynamics
Deep Differentiable Reinforcement Learning for Generic Energy Storage Day-Ahead Bidding with Transformer Neural Networks

Jinyu Liu,Hongye Guo,Qixin Chen

DOI: https://doi.org/10.1109/pesgm51994.2024.10688380

2024-01-01

Abstract:The increasing prevalence of renewable energies has led to greater volatility in electricity prices, posing a greater challenge for Energy Storage (ES) to arbitrage effectively in the fluctuating spot markets. To better cope with price uncertainties using large amounts of hidden experiences from global electricity markets, we propose an approach called Transformer-DDRL to enhance the performance of Day-Ahead (DA) ES bidding. Initially, based on the Transformer network, we develop an expressive deep neural network that is capable of encoding large amounts of bidding experiences and generating 24-hour DA bidding curves. Subsequently, we train the proposed Transformer network based on Deep Differentiable Reinforcement Learning (DDRL). Using DDRL, we directly optimize the network parameters of the Transformer with respect to the bidding objective to achieve effective learning. Finally, we train the proposed bidder on a global price dataset and achieve state-of-the-art returns in multiple markets, demonstrating its promise as a versatile market bidding model.
ReLeDP: Reinforcement-Learning-Assisted Dynamic Pricing for Wireless Smart Grid

Peiying Zhang,Chao Wang,Gagangeet Singh Aujla,Ranbir Singh Batth

DOI: https://doi.org/10.1109/mwc.011.2000431

IF: 12.777

2021-12-01

IEEE Wireless Communications

Abstract:The smart grid must ensure that power providers can obtain substantial benefits by selling energy, while at the same time, they need to consider the cost of consumers. To realize this win-win situation, the smart grid relies on dynamic pricing mechanisms. However, most of the existing dynamic pricing schemes are based on artificial objective rules or conventional models, which cannot ensure the desired effectiveness. Thus, we apply reinforcement learning to model the supply-demand relationship between power providers and consumers in a smart grid. The dynamic pricing problem of the smart grid is modeled as a discrete Markov decision process, and the decision process is solved by Q-learning. Now, the success of any intelligent dynamic pricing scheme relies on timely data transmission. However, the scale and speed of data generation can create several network bottlenecks that can further reduce the performance of any dynamic pricing scheme. Hence, to overcome this challenge, we have proposed an artificial-intelligence-based adaptive network architecture that adopts software-defined networking. In this architecture, we have used a self-organized map-based traffic classification approach followed by a dynamic virtual network embedding mechanism. We demonstrate the effectiveness of the dynamic pricing strategy supported through adaptive network architecture based on various performance indicators. The outcomes suggest that the proposed strategy is of great significance to realize the sustainability of power energy in the future. Lastly, we discuss various implementation challenges and future directions before concluding the article.

computer science, information systems,telecommunications,engineering, electrical & electronic, hardware & architecture
Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets

Jinhao Li,Changlong Wang,Yanru Zhang,Hao Wang

2024-02-29

Abstract:The battery energy storage system (BESS) has immense potential for enhancing grid reliability and security through its participation in the electricity market. BESS often seeks various revenue streams by taking part in multiple markets to unlock its full potential, but effective algorithms for joint-market participation under price uncertainties are insufficiently explored in the existing research. To bridge this gap, we develop a novel BESS joint bidding strategy that utilizes deep reinforcement learning (DRL) to bid in the spot and contingency frequency control ancillary services (FCAS) markets. Our approach leverages a transformer-based temporal feature extractor to effectively respond to price fluctuations in seven markets simultaneously and helps DRL learn the best BESS bidding strategy in joint-market participation. Additionally, unlike conventional "black-box" DRL model, our approach is more interpretable and provides valuable insights into the temporal bidding behavior of BESS in the dynamic electricity market. We validate our method using realistic market prices from the Australian National Electricity Market. The results show that our strategy outperforms benchmarks, including both optimization-based and other DRL-based strategies, by substantial margins. Our findings further suggest that effective temporal-aware bidding can significantly increase profits in the spot and contingency FCAS markets compared to individual market participation.

Systems and Control,Machine Learning,Optimization and Control
Data-driven online interactive bidding strategy for demand response

Kuan-Cheng Lee,Hong-Tzer Yang,Wenjun Tang

DOI: https://doi.org/10.1016/j.apenergy.2022.119082

IF: 11.2

2022-08-01

Applied Energy

Abstract:Demand response (DR), as one of the important energy resources in the future’s grid, provides the services of peak shaving, enhancing the efficiency of renewable energy utilization with a short response period, and low cost. Various categories of DR are established, e.g. automated DR, incentive DR, emergency DR, and demand bidding. However, with the practical issue of the unawareness of residential and commercial consumers’ utility models, the researches about demand bidding aggregator involved in the electricity market are just at the beginning stage. For this issue, the bidding price and bidding quantity are two required decision variables while considering the uncertainties due to the market and participants. In this paper, we determine the bidding and purchasing strategy simultaneously employing the smart meter data and functions. A two-agent deep deterministic policy gradient method is developed to optimize the decisions through learning historical bidding experiences. The online learning further utilizes the daily newest bidding experience attained to ensure trend tracing and self-adaptation. Two environment simulators are adopted for testifying the robustness of the model. The results prove that when facing diverse situations the proposed model can earn the optimal profit via off/online learning the bidding rules and robustly making the proper bid.

energy & fuels,engineering, chemical
Finding individual strategies for storage units in electricity market models using deep reinforcement learning

Nick Harder,Anke Weidlich,Philipp Staudt

DOI: https://doi.org/10.1186/s42162-023-00293-0

2023-10-19

Energy Informatics

Abstract:Abstract Modeling energy storage units realistically is challenging as their decision-making is not governed by a marginal cost pricing strategy but relies on expected electricity prices. Existing electricity market models often use centralized rule-based bidding or global optimization approaches, which may not accurately capture the competitive behavior of market participants. To address this issue, we present a novel method using multi-agent deep reinforcement learning to model individual strategies in electricity market models. We demonstrate the practical applicability of our approach using a detailed model of the German wholesale electricity market with a complete fleet of pumped hydro energy storage units represented as learning agents. We compare the results to widely used modeling approaches and demonstrate that the proposed method performs well and can accurately represent the competitive behavior of market participants. To understand the benefits of using reinforcement learning, we analyze overall profits, aggregated dispatch, and individual behavior of energy storage units. The proposed method can improve the accuracy and realism of electricity market modeling and help policymakers make informed decisions for future market designs and policies.
Bidding Strategic of Virtual Power Plant Based on End-to-End Deep Reinforcement Learning

Jiaxuan Li,Ye Guo,Hongbin Sun,Xin Wen

DOI: https://doi.org/10.1109/ei259745.2023.10512616

2023-01-01

Abstract:Apply deep reinforcement learning for bidding strategy generation in electricity market, enables the market participant to obtain accurate feedback with the bidding action on the impact of market clearing outcomes, then optimize the action to be more profitable according to the feedback. However, current deep reinforcement learning methods focusing on handcrafted states, such as a fixed number with confidence coefficient of the generation amount, which limited the capability of dealing with virtual power plant bidding with uncertainty renewable energy generation amount. This paper uses the pretext tasks to pre-train the feature extractors, but omits the task headers in bidding task to keep more information, instead of compressing the information to a number. This paper also proposes an end-to-end method, which takes the historical data as the input and outputs the action. Leveraging feature extraction networks to discover the information of uncertainty energy in the end-to-end method, then using the deep reinforcement learning method for bidding action generation. Our method outperforms methods using regression model to predict the generation and put the prediction result to states with an average increasement of 37.57% per day.
Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review

Lefeng Cheng,Xin Wei,Manling Li,Can Tan,Meng Yin,Teng Shen,Tao Zou

DOI: https://doi.org/10.3390/math12203241

IF: 2.4

2024-10-17

Mathematics

Abstract:With the rapid development of smart grids, the strategic behavior evolution in user-side electricity market transactions has become increasingly complex. To explore the dynamic evolution mechanisms in this area, this paper systematically reviews the application of evolutionary game theory in user-side electricity markets, focusing on its unique advantages in modeling multi-agent interactions and dynamic strategy optimization. While evolutionary game theory excels in explaining the formation of long-term stable strategies, it faces limitations when dealing with real-time dynamic changes and high-dimensional state spaces. Thus, this paper further investigates the integration of deep reinforcement learning, particularly the deep Q-learning network (DQN), with evolutionary game theory, aiming to enhance its adaptability in electricity market applications. The introduction of the DQN enables market participants to perform adaptive strategy optimization in rapidly changing environments, thereby more effectively responding to supply–demand fluctuations in electricity markets. Through simulations based on a multi-agent model, this study reveals the dynamic characteristics of strategy evolution under different market conditions, highlighting the changing interaction patterns among participants in complex market environments. In summary, this comprehensive review not only demonstrates the broad applicability of evolutionary game theory in user-side electricity markets but also extends its potential in real-time decision making through the integration of modern algorithms, providing new theoretical foundations and practical insights for future market optimization and policy formulation.

mathematics
A data‐driven method for microgrid bidding optimization in electricity market

Rudai Yan,Yan Xu

DOI: https://doi.org/10.1049/enc2.12093

2023-08-18

Energy Conversion and Economics

Abstract:This paper presents a deep reinforcement learning based data‐driven solution to the microgrid bidding in the electricity market considering offers for the reserve market. The framework, based on the Markov decision process, models the microgrid's participation in the electricity market at different stages, including bidding, market‐clearing, and reserve activation. The problem is split into two stages: day‐ahead submission and real‐time market period, and the proposed method mainly focus on the first stage. The state information from state‐space models of distributed energy resources serves as input for the policy network. A deep deterministic policy gradient is employed to train the network and produce a deterministic bidding strategy. The second stage can then adjust this strategy based on the results from the first stage. The method is validated with real‐world microgrid systems and data from the Singapore spot market.
Dynamic Matching Markets in Power Grid: Concepts and Solution using Deep Reinforcement Learning

Majid Majidi,Deepan Muthirayan,Masood Parvania,Pramod P. Khargonekar

DOI: https://doi.org/10.48550/arXiv.2104.05654

2021-12-17

Abstract:Traditional bulk load flexibility options, such as load shifting and load curtailment, for managing uncertainty in power markets limit the diversity of options and ignore the preferences of the individual loads, thus reducing efficiency and welfare. This paper proposes an alternative to bulk load flexibility options for managing uncertainty in power markets: a reinforcement learning based dynamic matching market. We propose a novel hybrid learning-based model for maximizing social welfare in the dynamic matching market. The key features of our model is a fixed rule-based function and a learnable component that can be trained by data gathered online with no prior knowledge or expert supervision. The output of the learnable component is a probability distribution over the matching decisions for the individual customers. The proposed hybrid model enables the learning algorithm to find an effective matching policy that simultaneously satisfies the customers' servicing preferences. The simulations show that the learning algorithm learns an effective matching policy for different generation-consumption profiles and exhibits better performance compared to standard online matching heuristics such as Match on Arrival, Match to the Highest, and Match to the Earliest Deadline policies.

Systems and Control
2A Reliable Energy Trading Strategy in Intelligent Microgrids Using Deep Reinforcement Learning

Man Cao,Zhiyong Yin,Yajun Wang,Le Yu,Peiran Shi,Zhi Cai

DOI: https://doi.org/10.1016/j.compeleceng.2023.108796

2023-06-16

Abstract:A microgrid is a system that incorporates various decentralized power sources and manages the distribution of electricity. In microgrid, prosumers play a significant role in trading electricity, and it is essential to establish an effective power trading mechanism to incentivize their active participation. This study proposes a power trading mechanism for prosumers in microgrids, which incorporates blockchain technology to protect their rights and interests. The mechanism employs reinforcement learning to optimize trading decisions and develops a reputation mechanism to evaluate prosumers' trustworthiness based on their past transactions. The proposed strategy encourages prosumers to conduct more transactions with honest peers by introducing reputation value rewards into the benefit function. The simulation results indicate that the proposed strategy outperforms traditional approaches and reputation value significantly impacts prosumers' utility. Overall, the proposed strategy aims to promote honest transactions and enhance prosumers' participation in microgrid power transactions.

engineering, electrical & electronic,computer science, interdisciplinary applications, hardware & architecture
Optimal demand response based dynamic pricing strategy via Multi-Agent Federated Twin Delayed Deep Deterministic policy gradient algorithm

Haining Ma,Huifeng Zhang,Ding Tian,Dong Yue,Gerhard P. Hancke

DOI: https://doi.org/10.1016/j.engappai.2024.108012

IF: 8

2024-02-07

Engineering Applications of Artificial Intelligence

Abstract:The intermittent integration of renewable energy sources and the enhancement of energy-saving awareness on the demand side have posed significant challenges to energy management in smart grids. To address the supply–demand imbalance issues caused by these challenges, this paper proposes a novel multiple time-scale dispatch model. The model utilizes demand response strategy for demand side management (DSM) and employs dynamic pricing strategy for supply-side management. Furthermore, an advanced approach based on federated learning and Twin Delayed Deep Deterministic(TD3) policy gradient algorithm is proposed to construct a federated learning mechanism among multi agents. This approach performs distributed optimization on the multi-agent system to derive the optimal network weights for multiple stakeholders, resulting in the generation of an optimal dynamic pricing strategy that ensures the economic efficiency and secure operation of the smart grid. From the methodological perspective, this approach enhances privacy protection and decision-making autonomy for each stakeholder, while improving the optimization efficiency of the reinforcement learning (RL) algorithm. Based on simulation results, it is evident that the proposed method facilitates economically efficient transactions between supply and demand, and the improved distributed multi-agent federated reinforcement learning (MAFRL) approach holds promise as a feasible and optimal optimization method for economic dispatch in smart grid.

automation & control systems,computer science, artificial intelligence,engineering, electrical & electronic, multidisciplinary
Dynamic Demand-Aware Power Grid Intelligent Pricing Algorithm Based on Deep Reinforcement Learning

Chao Tang,Yunchuan Qin,Fan Wu,Zhuo Tang

DOI: https://doi.org/10.1109/access.2024.3406338

IF: 3.9

2024-06-04

IEEE Access

Abstract:The increasing integration of renewable energy sources and the growing complexity of modern power grids demand innovative solutions for efficient energy management. This paper introduces a novel dynamic demand-aware Power Grid Intelligent Pricing (PGIP) algorithm based on Deep Reinforcement Learning (DRL). The proposed PGIP algorithm aims to optimize energy consumption and pricing in real time by leveraging the capabilities of DRL to adapt to dynamic demand patterns and evolving grid conditions. PGIP employs a sophisticated neural network architecture to model the intricate relationships between various grid parameters, user demand, and pricing strategies. Through continuous learning and adaptation, the algorithm dynamically adjusts pricing structures to incentivize demand-side flexibility while ensuring grid stability. The reinforcement learning framework enables the algorithm to discover optimal policies for pricing in response to changing environmental factors and user behaviors. We used real-world data sets to assess its performance in diverse scenarios. Results demonstrate the algorithm's ability to optimize energy consumption, reduce peak demand, and enhance overall grid efficiency. Moreover, comparisons with traditional pricing models highlight the superior adaptability and responsiveness of PGIP in addressing the challenges posed by the evolving landscape of power grids. PGIP presents a promising approach to address the dynamic nature of power grids and the increasing demand for efficient energy management.

computer science, information systems,telecommunications,engineering, electrical & electronic

Re: "Management of Marcus Gunn jaw winking synkinesis".

Deep Reinforcement Learning for Strategic Bidding in Electricity Markets

Deep reinforcement learning-based optimal bidding strategy for real-time multi-participant electricity market with short-term load

Deep Reinforcement Learning Based Pricing Strategy of Aggregators Considering Renewable Energy

Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity

Blockchain-Assisted Secure Energy Trading in Electricity Markets: A Tiny Deep Reinforcement Learning-Based Stackelberg Game Approach

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

A Deep Reinforcement Learning Bidding Algorithm on Electricity Market

Deep Differentiable Reinforcement Learning for Generic Energy Storage Day-Ahead Bidding with Transformer Neural Networks

ReLeDP: Reinforcement-Learning-Assisted Dynamic Pricing for Wireless Smart Grid

Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets

Data-driven online interactive bidding strategy for demand response

Finding individual strategies for storage units in electricity market models using deep reinforcement learning

Bidding Strategic of Virtual Power Plant Based on End-to-End Deep Reinforcement Learning

Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review

A data‐driven method for microgrid bidding optimization in electricity market

Dynamic Matching Markets in Power Grid: Concepts and Solution using Deep Reinforcement Learning

2A Reliable Energy Trading Strategy in Intelligent Microgrids Using Deep Reinforcement Learning

Optimal demand response based dynamic pricing strategy via Multi-Agent Federated Twin Delayed Deep Deterministic policy gradient algorithm

Dynamic Demand-Aware Power Grid Intelligent Pricing Algorithm Based on Deep Reinforcement Learning