Abstract:With large-scale grid integration of renewable energy sources (RES), power grid operations gradually exhibit the new characteristics of high-order uncertainty, leading to significant challenges for system operational security. Traditional model-driven generation dispatch methods require large computational resources, whereas the widely concerned Reinforcement Learning (RL)-based methods lead to issues such as slow training speed due to the high complexity and dimension of processed grid state information. For this reason, this paper proposes a novel Grid Expert Strategy Imitation Learning (GESIL)-based real-time (5 min intervals in this paper) dispatch method. Firstly, a grid model is established based on the graph theory. Secondly, a pure rule-based grid expert strategy (GES) considering detailed power grid operations is proposed. Then, the GES is combined with the established model to obtain a GESIL agent using imitation learning by offline–online training, which can produce specific grid dispatch decisions for real-time. By designing a graph theory-based grid model, a model-driven purely rule-based GES, and embedding a penalty factor-based loss function into IL offline–online training, GESIL ultimately achieves high training speed, high solution speed, and strong generalization capability. A modified IEEE 118-node system is employed to compare the proposed GESIL to traditional dispatch method and RL method. Results show that GESIL has significantly improved computational efficiency by approximately 17 times and training speed by 14.5 times. GESIL can more stably and efficiently compute real-time dispatch decisions of grid operations, enhancing the optimization effect in terms of transmission overloading mitigation, transmission loading optimization, and power balancing control.

Real-time Power Scheduling Through Reinforcement Learning from Demonstrations

Real-time scheduling of renewable power systems through planning-based reinforcement learning

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

Real-Time Scheduling of High-Penetrated Renewable Power Systems: an Expert Knowledge and Reinforcement Learning Hybrid Approach

Physics-Informed Reinforcement Learning for Real-Time Optimal Power Flow with Renewable Energy Resources

Real-Time Optimal Power Flow Method Via Safe Deep Reinforcement Learning Based on Primal-Dual and Prior Knowledge Guidance

Online Optimal Power Scheduling of a Microgrid via Imitation Learning

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

On Training Effective Reinforcement Learning Agents for Real-time Power Grid Operation and Control

A Graph-based Deep Reinforcement Learning Framework for Autonomous Power Dispatch on Power Systems with Changing Topologies

Online Scheduling of a Residential Microgrid via Monte-Carlo Tree Search and a Learned Model

Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm

Real-time Power System Dispatch Scheme Using Grid Expert Strategy-Based Imitation Learning

Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control

Real-Time Scheduling Method Based on Deep Reinforcement Learning for a Hybrid Wind-Solar-Storage Energy Generation System

Hierarchical deep reinforcement learning for self-adaptive economic dispatch

Reinforcement Learning for Resilient Power Grids

Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices