Abstract:Demand response improves grid security by adjusting the flexibility of consumers meanwhile maintaining their demand–supply balance in real-time. With the large-scale deployment of distributed digital communication technologies and advanced metering infrastructures, data-driven approaches such as multi-agent reinforcement learning (MARL) are being widely employed to solve demand response problems. Nevertheless, the massive interaction of data inside and outside the demand response management system may lead to severe threats from the perspective of cyber-attacks. The cyber security requirements of MARL-based demand response problems are less discussed in the existing studies. To this end, this paper proposes a robust adversarial multi-agent reinforcement learning framework for demand response (RAMARL-DR) with an enhanced resilience against adversarial attacks. In particular, the proposed RAMARL-DR first constructs an adversary agent that aims to cause the worst-case performance via formulating an adversarial attack; and then adopts periodic alternating robust adversarial training scenarios with the optimal adversary aiming to diminish the severe impacts induced by adversarial attacks. Case studies are conducted based on an OpenAI Gym environment CityLearn, which provides a standard evaluation platform of MARL algorithms for demand response problems. Empirical results indicate that the MARL-based demand response management system is vulnerable when the adversary agent occurs, and its performance can be significantly improved after periodic alternating robust adversarial training. It can be found that the adversary agent can result in a 41.43% higher metric value of Ramping than the no adversary case, whereas the proposed RAMARL-DR can significantly enhance the system resilience with an approximately 38.85% reduction in the ramping of net demand.

Enhancing Cyber-Resilience in Integrated Energy System Scheduling with Demand Response Using Deep Reinforcement Learning

Resilience Enhancement of Multi-Agent Reinforcement Learning-Based Demand Response Against Adversarial Attacks

Exploring the Vulnerability of Deep Reinforcement Learning-based Emergency Control for Low Carbon Power Systems

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

An Integrated Demand Response-Based Energy Management Strategy for Integrated Energy System Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Distribution System Cyber Attack Defense with DERs.

Soft Actor-Critic DRL Algorithm for Interval Optimal Dispatch of Integrated Energy Systems with Uncertainty in Demand Response and Renewable Energy

Deep Reinforcement Learning for Cybersecurity Assessment of Wind Integrated Power Systems

Physics-Constrained Vulnerability Assessment of Deep Reinforcement Learning-Based SCOPF

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

A deep reinforcement learning-based resilience enhancement framework for distribution networks under extreme weather events

A Reinforcement Learning Engine with Reduced Action and State Space for Scalable Cyber-Physical Optimal Response

Online Operational Decision-making for Integrated Electric-Gas Systems with Safe Reinforcement Learning

Stochastic Planning for Resilient Infrastructure of Distribution System under Extreme Weather Events

Integrated Demand Response in Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach

A Real-time Demand Response Strategy of Home Energy Management by Using Distributed Deep Reinforcement Learning.

Real-Time Optimal Power Flow Method Via Safe Deep Reinforcement Learning Based on Primal-Dual and Prior Knowledge Guidance

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

Towards Risk-Aware Real-Time Security Constrained Economic Dispatch: A Tailored Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Optimal Scheduling of Active Distribution System Considering Distributed Generation, Energy Storage and Flexible Load

Enhancing the Tolerance of Voltage Regulation to Cyber Contingencies Via Graph-Based Deep Reinforcement Learning