Abstract:The decarbonization of energy systems has posed unprecedented challenges in system complexity and operational uncertainty that render it imperative to exploit cutting-edge artificial intelligence (AI) technologies to realize real-time, autonomous power system operation and control. In particular, deep reinforcement learning (DRL)-based approaches in power systems are extensively studied and implemented in several trials worldwide. Nevertheless, the vulnerability of DRL brings new security threats to power systems that have not been well identified and investigated in the literature. To this end, this paper proposes a physics-constrained vulnerability assessment methodological framework for the DRL-based power system operation and control, with a special focus on the problem of security-constrained optimal power flow (SCOPF). In particular, we develop a novel adversarial example generation method, defined as a false data injection attack against the DRL-based SCOPF (FDIAI), to realize a targeted adversarial attack considering the nonlinear physical constraints in power systems via two main stages of constructor function design and unconstrained optimization problem transformation. In this way, the proposed FDIAI can significantly influence the decision-making procedure of DRL while successfully evading the bad data detection mechanism in power systems. Case studies are conducted to explore the stealthiness and effectiveness of FDIAI and then show its severe impacts on system operation and control on the winners' models of the Learning to Run a Power Network (L2RPN) competitions, including L2RPN IJCNN 2019 (IJCNN), L2RPN WCCI 2020 (WCCI), and L2RPN NeurIPS 2020 (NeurIPS).

Robust preventive and corrective security-constrained OPF for worst contingencies with the adoption of VPP: A safe reinforcement learning approach

Robust N - k Security-constrained Optimal Power Flow Incorporating Preventive and Corrective Generation Dispatch to Improve Power System Reliability

Physics-Constrained Vulnerability Assessment of Deep Reinforcement Learning-Based SCOPF

Safe deep reinforcement learning for real-time AC optimal power flow: A near-optimal solution

An efficient approach for robust SCOPF considering load and renewable power uncertainties

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

Real-Time Optimal Power Flow Method Via Safe Deep Reinforcement Learning Based on Primal-Dual and Prior Knowledge Guidance

Improved PSCOPF model based on risk management and alternative iteration algorithm

Preventive Security-Constrained Optimal Power Flow with Probabilistic Guarantees

Adjustable Robust Optimization for Corrective Optimal Power Flow under N-1 Contingencies

Solving Preventive-Corrective SCOPF by a Hybrid Computational Strategy

Risk-Oriented Preventive Control of Transmission Lines Overload

Security-constrained optimal power flow based on risk ranking of contingencies

Online Preventive Control for Transmission Overload Relief Using Safe Reinforcement Learning with Enhanced Spatial-Temporal Awareness

Preventive Security-Constrained Optimal Power Flow with Constraints Limiting the Number of Beforehand Control Actions

Adjustable Robust Optimal Power Flow with the Price of Robustness for Large-Scale Power Systems

Real-time Sequential Security-Constrained Optimal Power Flow: A Hybrid Knowledge-Data-Driven Reinforcement Learning Approach

Computationally Enhanced Approach for Chance-Constrained OPF Considering Voltage Stability

Closure to Discussion on “Solving Preventive-Corrective SCOPF by a Hybrid Computational Strategy”

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control