Abstract:Demand response improves grid security by adjusting the flexibility of consumers meanwhile maintaining their demand–supply balance in real-time. With the large-scale deployment of distributed digital communication technologies and advanced metering infrastructures, data-driven approaches such as multi-agent reinforcement learning (MARL) are being widely employed to solve demand response problems. Nevertheless, the massive interaction of data inside and outside the demand response management system may lead to severe threats from the perspective of cyber-attacks. The cyber security requirements of MARL-based demand response problems are less discussed in the existing studies. To this end, this paper proposes a robust adversarial multi-agent reinforcement learning framework for demand response (RAMARL-DR) with an enhanced resilience against adversarial attacks. In particular, the proposed RAMARL-DR first constructs an adversary agent that aims to cause the worst-case performance via formulating an adversarial attack; and then adopts periodic alternating robust adversarial training scenarios with the optimal adversary aiming to diminish the severe impacts induced by adversarial attacks. Case studies are conducted based on an OpenAI Gym environment CityLearn, which provides a standard evaluation platform of MARL algorithms for demand response problems. Empirical results indicate that the MARL-based demand response management system is vulnerable when the adversary agent occurs, and its performance can be significantly improved after periodic alternating robust adversarial training. It can be found that the adversary agent can result in a 41.43% higher metric value of Ramping than the no adversary case, whereas the proposed RAMARL-DR can significantly enhance the system resilience with an approximately 38.85% reduction in the ramping of net demand.

Multi-Agent Deep Reinforcement Learning-Driven Mitigation of Adverse Effects of Cyber-Attacks on Electric Vehicle Charging Station

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

Exploring the Vulnerability of Deep Reinforcement Learning-based Emergency Control for Low Carbon Power Systems

Resilience Enhancement of Multi-Agent Reinforcement Learning-Based Demand Response Against Adversarial Attacks

Charging Ahead: A Hierarchical Adversarial Framework for Counteracting Advanced Cyber Threats in EV Charging Stations

Deep Reinforcement Learning for DER Cyber-Attack Mitigation

An Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control

Centralized vs. Decentralized Multi-Agent Reinforcement Learning for Enhanced Control of Electric Vehicle Charging Networks

Deep Reinforcement Learning for Distribution System Cyber Attack Defense with DERs.

Physics-Shielded Multi-Agent Deep Reinforcement Learning for Safe Active Voltage Control with Photovoltaic/Battery Energy Storage Systems

Charge Manipulation Attacks Against Smart Electric Vehicle Charging Stations and Deep Learning-based Detection Mechanisms

Data-driven load frequency cooperative control for multi-area power system integrated with VSCs and EV aggregators under cyber-attacks

Exploring cybersecurity issues in 5G enabled electric vehicle charging station with deep learning

DRL2FC: An Attack-Resilient Controller for Automatic Generation Control Based on Deep Reinforcement Learning

EV Charging Strategy Considering Transformer Lifetime Via Evolutionary Curriculum Learning-Based Multiagent Deep Reinforcement Learning

Optimization of Mitigation Deployment Using Deep Reinforcement Learning over an Enhanced ATT CK

An IoT-Based Deep-Learning Architecture to Secure Automated Electric Vehicles Against Cyberattacks and Data Loss

Smart Grid Security: A Verified Deep Reinforcement Learning Framework to Counter Cyber-Physical Attacks

Protecting the Future Grid: An Electric Vehicle Robust Mitigation Scheme Against Load Altering Attacks on Power Grids

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks