Abstract:The decarbonization of energy systems has posed unprecedented challenges in system complexity and operational uncertainty that render it imperative to exploit cutting-edge artificial intelligence (AI) technologies to realize real-time, autonomous power system operation and control. In particular, deep reinforcement learning (DRL)-based approaches in power systems are extensively studied and implemented in several trials worldwide. Nevertheless, the vulnerability of DRL brings new security threats to power systems that have not been well identified and investigated in the literature. To this end, this paper proposes a physics-constrained vulnerability assessment methodological framework for the DRL-based power system operation and control, with a special focus on the problem of security-constrained optimal power flow (SCOPF). In particular, we develop a novel adversarial example generation method, defined as a false data injection attack against the DRL-based SCOPF (FDIAI), to realize a targeted adversarial attack considering the nonlinear physical constraints in power systems via two main stages of constructor function design and unconstrained optimization problem transformation. In this way, the proposed FDIAI can significantly influence the decision-making procedure of DRL while successfully evading the bad data detection mechanism in power systems. Case studies are conducted to explore the stealthiness and effectiveness of FDIAI and then show its severe impacts on system operation and control on the winners' models of the Learning to Run a Power Network (L2RPN) competitions, including L2RPN IJCNN 2019 (IJCNN), L2RPN WCCI 2020 (WCCI), and L2RPN NeurIPS 2020 (NeurIPS).

A Data-driven Method for Fast AC Optimal Power Flow Solutions via Deep Reinforcement Learning

Safe deep reinforcement learning for real-time AC optimal power flow: A near-optimal solution

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

Physics-Informed Reinforcement Learning for Real-Time Optimal Power Flow with Renewable Energy Resources

Data-driven AC Optimal Power Flow with Physics-informed Learning and Calibrations

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

On Fast-Converged Deep Reinforcement Learning for Optimal Dispatch of Large-Scale Power Systems under Transient Security Constraints

On Training Effective Reinforcement Learning Agents for Real-time Power Grid Operation and Control

Physics-Constrained Vulnerability Assessment of Deep Reinforcement Learning-Based SCOPF

Learning Optimal Solutions for Extremely Fast AC Optimal Power Flow

Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control

Constrained Reinforcement Learning for Predictive Control in Real-Time Stochastic Dynamic Optimal Power Flow

Deep reinforcement learning based power system optimal carbon emission flow

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

Implementing Deep Reinforcement Learning-Based Grid Voltage Control in Real-World Power Systems: Challenges and Insights

Unsupervised Learning for Solving AC Optimal Power Flows: Design, Analysis, and Experiment

Real-time Sequential Security-Constrained Optimal Power Flow: A Hybrid Knowledge-Data-Driven Reinforcement Learning Approach

Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning

OptNet-Embedded Data-Driven Approach for Optimal Power Flow Proxy