Abstract:The decarbonization of energy systems has posed unprecedented challenges in system complexity and operational uncertainty that render it imperative to exploit cutting-edge artificial intelligence (AI) technologies to realize real-time, autonomous power system operation and control. In particular, deep reinforcement learning (DRL)-based approaches in power systems are extensively studied and implemented in several trials worldwide. Nevertheless, the vulnerability of DRL brings new security threats to power systems that have not been well identified and investigated in the literature. To this end, this paper proposes a physics-constrained vulnerability assessment methodological framework for the DRL-based power system operation and control, with a special focus on the problem of security-constrained optimal power flow (SCOPF). In particular, we develop a novel adversarial example generation method, defined as a false data injection attack against the DRL-based SCOPF (FDIAI), to realize a targeted adversarial attack considering the nonlinear physical constraints in power systems via two main stages of constructor function design and unconstrained optimization problem transformation. In this way, the proposed FDIAI can significantly influence the decision-making procedure of DRL while successfully evading the bad data detection mechanism in power systems. Case studies are conducted to explore the stealthiness and effectiveness of FDIAI and then show its severe impacts on system operation and control on the winners' models of the Learning to Run a Power Network (L2RPN) competitions, including L2RPN IJCNN 2019 (IJCNN), L2RPN WCCI 2020 (WCCI), and L2RPN NeurIPS 2020 (NeurIPS).

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

Real-Time Optimal Power Flow Method Via Safe Deep Reinforcement Learning Based on Primal-Dual and Prior Knowledge Guidance

Safe deep reinforcement learning for real-time AC optimal power flow: A near-optimal solution

A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

Physics-Informed Reinforcement Learning for Real-Time Optimal Power Flow with Renewable Energy Resources

On Fast-Converged Deep Reinforcement Learning for Optimal Dispatch of Large-Scale Power Systems under Transient Security Constraints

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Physics-Constrained Vulnerability Assessment of Deep Reinforcement Learning-Based SCOPF

Real-time Autonomous Line Flow Control Using Proximal Policy Optimization

Constrained Reinforcement Learning for Predictive Control in Real-Time Stochastic Dynamic Optimal Power Flow

Real-time Sequential Security-Constrained Optimal Power Flow: A Hybrid Knowledge-Data-Driven Reinforcement Learning Approach

Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning

Deep Reinforcement Learning Based Approach for Optimal Power Flow of Distribution Networks Embedded with Renewable Energy and Storage Devices

Deep Reinforcement Learning for Optimal Power Flow with Renewables Using Graph Information

Towards Risk-Aware Real-Time Security Constrained Economic Dispatch: A Tailored Deep Reinforcement Learning Approach

A General Real-time OPF Algorithm Using DDPG with Multiple Simulation Platforms

Distributed Deep Reinforcement Learning-based Approach for Fast Preventive Control Considering Transient Stability Constraints

Power balance control of RES integrated power system by deep reinforcement learning with optimized utilization rate of renewable energy

A Graph-based Deep Reinforcement Learning Framework for Autonomous Power Dispatch on Power Systems with Changing Topologies

Robust preventive and corrective security-constrained OPF for worst contingencies with the adoption of VPP: A safe reinforcement learning approach

Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm