Abstract:The decarbonization of energy systems has posed unprecedented challenges in system complexity and operational uncertainty that render it imperative to exploit cutting-edge artificial intelligence (AI) technologies to realize real-time, autonomous power system operation and control. In particular, deep reinforcement learning (DRL)-based approaches in power systems are extensively studied and implemented in several trials worldwide. Nevertheless, the vulnerability of DRL brings new security threats to power systems that have not been well identified and investigated in the literature. To this end, this paper proposes a physics-constrained vulnerability assessment methodological framework for the DRL-based power system operation and control, with a special focus on the problem of security-constrained optimal power flow (SCOPF). In particular, we develop a novel adversarial example generation method, defined as a false data injection attack against the DRL-based SCOPF (FDIAI), to realize a targeted adversarial attack considering the nonlinear physical constraints in power systems via two main stages of constructor function design and unconstrained optimization problem transformation. In this way, the proposed FDIAI can significantly influence the decision-making procedure of DRL while successfully evading the bad data detection mechanism in power systems. Case studies are conducted to explore the stealthiness and effectiveness of FDIAI and then show its severe impacts on system operation and control on the winners' models of the Learning to Run a Power Network (L2RPN) competitions, including L2RPN IJCNN 2019 (IJCNN), L2RPN WCCI 2020 (WCCI), and L2RPN NeurIPS 2020 (NeurIPS).

Risk-based Dispatch of Power Systems Incorporating Spatiotemporal Correlation Based on the Robust Soft Actor-Critic Algorithm

Multi-Agent Reinforcement Learning-based Distributed Economic Dispatch Considering Network attacks and Uncertain Costs

Towards Risk-Aware Real-Time Security Constrained Economic Dispatch: A Tailored Deep Reinforcement Learning Approach

Soft Actor-Critic DRL Algorithm for Interval Optimal Dispatch of Integrated Energy Systems with Uncertainty in Demand Response and Renewable Energy

Reinforcement Learning with Enhanced Safety for Optimal Dispatch of Distributed Energy Resources in Active Distribution Networks

Physics-Constrained Vulnerability Assessment of Deep Reinforcement Learning-Based SCOPF

A Safe DRL Method for Fast Solution of Real-Time Optimal Power Flow

Distributionally Robust Dynamic Economic Dispatch Model with Conditional Value at Risk Recourse Function

Synergistic Risk-Based Robust Scheduling for Hybrid AC-DC Systems via Analytical Target Cascading

A Low-Carbon Economic Dispatch Method for Power Systems with Carbon Capture Plants Based on Safe Reinforcement Learning

Robust preventive and corrective security-constrained OPF for worst contingencies with the adoption of VPP: A safe reinforcement learning approach

Rethinking Safe Policy Learning for Complex Constraints Satisfaction: A Glimpse in Real-Time Security Constrained Economic Dispatch Integrating Energy Storage Units

Fully Distributed Risk-Based Robust Multi-Area Dynamic Economic Dispatch

Fully Distributed Risk-based Robust Reserve Scheduling for Bulk Hybrid AC-DC Systems

On Fast-Converged Deep Reinforcement Learning for Optimal Dispatch of Large-Scale Power Systems under Transient Security Constraints

Real-Time Optimal Power Flow Method Via Safe Deep Reinforcement Learning Based on Primal-Dual and Prior Knowledge Guidance

Recourse-Cost Constrained Robust Optimization for Microgrid Dispatch With Correlated Uncertainties

Risk-based Security-Constrained Economic Dispatch in Power Systems

Two-Stage Stochastic Generation Dispatching Model and Method Considering Conditional Value-at-Risk

Risk-based Distributionally Robust Optimal Dispatch for Multiple Cascading Failures in Regional Integrated Energy System Using Surrogate Modeling

Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning