Abstract:In this paper, a novel hierarchical reinforcement learning (HRL) architecture is put forward to resolve the multi-area economic dispatch (MAED) problem in a fully off-line optimization manner, while the focus of MAED is to minimize the total generation cost of the entire multi-area power system by determining output power within area and the power transfer among areas. The original MAED problem is transformed into an augmented sub-MAED problem by exploring the relationship between power balance constraint and tie line capacity constraint in each area. This architecture first calculates the scheduled area load in the top layer through the primal–dual consensus-based distributed Q-learning, and then calculates load-power table assembled by the optimal output power within single area according to the multi-stage decision algorithm in the bottom layer. Acting as an underlying framework, the assigned load-power table is implemented with the combination of off-line calculation and on-line inquiry. It is capable of parallel and asynchronous optimization in each area while satisfying various constraints. Furthermore, a power-triggered algorithm is proposed to satisfy the power balance among areas, and its convergence is theoretically analyzed, which illustrates that the area power error will converge to zero in the limited iteration. Finally, the proposed approach is validated in three different cases: 20-unit, 40-unit, and 120-unit. The comparative results confirmed the superiority of the proposed HRL architecture with high real-time performance and less computation burden. Specifically, the HRL architecture reduces the fuel cost in the range of 0.01656%–0.19173% and accelerates the speed more than 10 times than others.

Stochastic Optimal Generation Command Dispatch Based on Improved Hierarchical Reinforcement Learning Approach

Hierarchical Control Strategy for Load Regulation Based on Stackelberg Game Theory Considering Randomness

Annual rolling generation scheduling and unit commitment optimization model based on open and impartial dispatching

Hierarchical deep reinforcement learning for self-adaptive economic dispatch

Data-Efficient Deep Reinforcement Learning-Based Optimal Generation Control in DC Microgrids

Offline economic dispatch for multi-area power system via hierarchical reinforcement learning

Application of Reinforcement Learning Method in a Hydro-thermal Hybrid Automatic Generation Control System

Power grid optimal topology dispatch and early fault warning using GAT and HDRL

Distributed Online Dispatch for Microgrids Using Hierarchical Reinforcement Learning Embedded with Operation Knowledge

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Reinforcement Learning and Stochastic Optimization with Deep Learning-Based Forecasting on Power Grid Scheduling

Closed-Loop Hierarchical Operation for Optimal Unit Commitment and Dispatch in Microgrids: A Hybrid System Approach

Day-ahead Scheduling Based on Reinforcement Learning with Hybrid Action Space

Research of coordinated optimal control strategy between units and grid based on hierarchical event-driving control

Optimal control in microgrid using multi-agent reinforcement learning

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

Hierarchical Generation Rescheduling and Robust Load Shedding Scheme Considering the Uncertainty of Distributed Generators

Online Optimal Generation Control Based on Constrained Distributed Gradient Algorithm

Multiagent-Based Reinforcement Learning for Optimal Reactive Power Dispatch.

A Cooperative Dispatch Algorithm for Hydrogen-Based Grid-Connection Microgrids: A Multi-Agent Reinforcement Learning Method

Distributed deep reinforcement learning for integrated generation‐control and power‐dispatch of interconnected power grid with various renewable units