Abstract:The large-scale integration of intermittent renewable energy resources introduces increased uncertainty and volatility to the supply side of power systems, thereby complicating system operation and control. Recently, data-driven approaches, particularly reinforcement learning (RL), have shown significant promise in addressing complex control challenges in power systems, because RL can learn from interactive feedback without needing prior knowledge of the system model. However, the training process of model-free RL methods relies heavily on random decisions for exploration, which may result in ``bad" decisions that violate critical safety constraints and lead to catastrophic control outcomes. Due to the inability of RL methods to theoretically ensure decision safety in power systems, directly deploying traditional RL algorithms in the real world is deemed unacceptable. Consequently, the safety issue in RL applications, known as safe RL, has garnered considerable attention in recent years, leading to numerous important developments. This paper provides a comprehensive review of the state-of-the-art safe RL techniques and discusses how these techniques can be applied to power system control problems such as frequency regulation, voltage control, and energy management. We then present discussions on key challenges and future research directions, related to convergence and optimality, training efficiency, universality, and real-world deployment.

Real-time Sequential Security-Constrained Optimal Power Flow: A Hybrid Knowledge-Data-Driven Reinforcement Learning Approach

Real-Time Optimal Power Flow Method Via Safe Deep Reinforcement Learning Based on Primal-Dual and Prior Knowledge Guidance

Physics-Informed Reinforcement Learning for Real-Time Optimal Power Flow with Renewable Energy Resources

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

Safe deep reinforcement learning for real-time AC optimal power flow: A near-optimal solution

Constrained Reinforcement Learning for Predictive Control in Real-Time Stochastic Dynamic Optimal Power Flow

A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

Optimal Operable Power Flow: Sample-efficient Holomorphic Embedding-based Reinforcement Learning

Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning

Real-time Autonomous Line Flow Control Using Proximal Policy Optimization

Deep Reinforcement Learning Based Approach for Active Power Security Correction Control of Power System

Deep Reinforcement Learning for Optimal Power Flow with Renewables Using Graph Information

Real-Time Scheduling of High-Penetrated Renewable Power Systems: an Expert Knowledge and Reinforcement Learning Hybrid Approach

On Fast-Converged Deep Reinforcement Learning for Optimal Dispatch of Large-Scale Power Systems under Transient Security Constraints

Deep Reinforcement Learning Based Approach for Optimal Power Flow of Distribution Networks Embedded with Renewable Energy and Storage Devices

Robust preventive and corrective security-constrained OPF for worst contingencies with the adoption of VPP: A safe reinforcement learning approach

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

Safe Reinforcement Learning for Power System Control: A Review

Real-time Power Scheduling Through Reinforcement Learning from Demonstrations

On Training Effective Reinforcement Learning Agents for Real-time Power Grid Operation and Control

Deep reinforcement learning based power system optimal carbon emission flow