Abstract:Recent works have revealed that backdoor attacks against Deep Reinforcement Learning (DRL) could lead to abnormal action selections of the agent, which may result in failure or even catastrophe in crucial decision processes. However, existing attacks only consider single-agent reinforcement learning (RL) systems, in which the only agent can observe the global state and have full control of the decision process. In this article, we explore a new backdoor attack paradigm in cooperative multi-agent reinforcement learning (CMARL) scenarios, where a group of agents coordinate with each other to achieve a common goal, while each agent can only observe the local state. In the proposed MARNet attack framework, we carefully design a pipeline of trigger design, action poisoning, and reward hacking modules to accommodate the cooperative multi-agent settings. In particular, as only a subset of agents can observe the triggers in their local observations, we maneuver their actions to the worst actions suggested by an expert policy model. Since the global reward in CMARL is aggregated by individual rewards from all agents, we propose to modify the reward in a way that boosts the bad actions of poisoned agents (agents who observe the triggers) but mitigates the influence on non-poisoned agents. We conduct extensive experiments on three classical CMARL algorithms VDN, COMA, and QMIX, in two popular CMARL games Predator Prey and SMAC. The results show that the baselines extended from single-agent DRL backdoor attacks seldom work in CMARL problems while MARNet performs well by reducing the utility under attack by nearly 100%. We apply fine-tuning as a potential defense against MARNet and demonstrate that fine-tuning cannot entirely eliminate the effect of the attack.

Towards Secure Multi-Agent Deep Reinforcement Learning: Adversarial Attacks and Countermeasures

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

Exploring the Vulnerability of Deep Reinforcement Learning-based Emergency Control for Low Carbon Power Systems

Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Security and Privacy Issues in Deep Reinforcement Learning: Threats and Countermeasures

Applying Reinforcement Learning for Enhanced Cybersecurity against Adversarial Simulation

Multi-Agent Guided Deep Reinforcement Learning Approach Against State Perturbed Adversarial Attacks

Characterizing Attacks on Deep Reinforcement Learning

A Deep Reinforcement Learning-Based Method Applied for Solving Multi-Agent Defense and Attack Problems.

Robustifying Reinforcement Learning Agents via Action Space Adversarial Training

Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey

Selective Real‐time Adversarial Perturbations Against Deep Reinforcement Learning Agents

Adversarial Attack and Defense in Reinforcement Learning-from AI Security View

Adversarial Attacks on Multiagent Deep Reinforcement Learning Models in Continuous Action Space

Multiple-Model Based Defense for Deep Reinforcement Learning Against Adversarial Attack

Deep Reinforcement Learning for Cybersecurity Applications

Robust Multi-Agent Reinforcement Learning against Adversaries on Observation

Deep multiagent reinforcement learning: challenges and directions

Reinforcement Learning-Based Approaches for Enhancing Security and Resilience in Smart Control: A Survey on Attack and Defense Methods

Adversarial Policies: Attacking Deep Reinforcement Learning