Abstract:The integration of renewable energy sources (RES) is rapidly increasing in electric power systems (EPS). While the inclusion of intermittent RES coupled with the wide-scale deployment of communication and sensing devices is important towards a fully smart grid, it has also expanded the cyber-threat landscape, effectively making power systems vulnerable to cyberattacks. This paper proposes a cybersecurity assessment approach designed to assess the cyberphysical security of EPS. The work takes into consideration the intermittent generation of RES, vulnerabilities introduced by microprocessor-based electronic information and operational technology (IT/OT) devices, and contingency analysis results. The proposed approach utilizes deep reinforcement learning (DRL) and an adapted Common Vulnerability Scoring System (CVSS) score tailored to assess vulnerabilities in EPS in order to identify the optimal attack transition policy based on N-2 contingency results, i.e., the simultaneous failure of two system elements. The effectiveness of the work is validated via numerical and real-time simulation experiments performed on literature-based power grid test cases. The results demonstrate how the proposed method based on deep Q-network (DQN) performs closely to a graph-search approach in terms of the number of transitions needed to find the optimal attack policy, without the need for full observation of the system. In addition, the experiments present the method's scalability by showcasing the number of transitions needed to find the optimal attack transition policy in a large system such as the Polish 2383 bus test system. The results exhibit how the proposed approach requires one order of magnitude fewer transitions when compared to a random transition policy.

Security Analysis of Cyber-Physical Systems Using Reinforcement Learning

A Hybrid Game Theory and Reinforcement Learning Approach for Cyber-Physical Systems Security

Secure State Estimation of Cyber-Physical System under Cyber Attacks: Q-Learning vs. SARSA

Exploring the Vulnerability of Deep Reinforcement Learning-based Emergency Control for Low Carbon Power Systems

Secure State Estimation of Cyber-Physical System under Cyber Attacks: <i>Q</i>-Learning vs. SARSA

Security Assessment of Industrial Control System Applying Reinforcement Learning

Deep Reinforcement Learning for Cybersecurity Assessment of Wind Integrated Power Systems

Smart Grid Security: A Verified Deep Reinforcement Learning Framework to Counter Cyber-Physical Attacks

Reinforcement Learning Solution for Cyber-Physical Systems Security Against Replay Attacks

A Reinforcement Learning Engine with Reduced Action and State Space for Scalable Cyber-Physical Optimal Response

An RL-Based Adaptive Detection Strategy to Secure Cyber-Physical Systems

A model-based approach for self-adaptive security in CPS : Application to smart grids

Crown Jewels Analysis using Reinforcement Learning with Attack Graphs

A review on machine learning techniques for secured cyber-physical systems in smart grid networks

Reinforcement Learning-Based Approaches for Enhancing Security and Resilience in Smart Control: A Survey on Attack and Defense Methods

A Systematic Analysis of Enhancing Cyber Security Using Deep Learning for Cyber Physical Systems

Deep Reinforcement Learning for Cybersecurity Applications

Exposing Surveillance Detection Routes via Reinforcement Learning, Attack Graphs, and Cyber Terrain

Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach

Security Algorithm for Intelligent Transport System in Cyber-Physical Systems Perceptive: Attacks, Vulnerabilities, and Countermeasures

Applying Reinforcement Learning for Enhanced Cybersecurity against Adversarial Simulation