Abstract:Recent works have revealed that backdoor attacks against Deep Reinforcement Learning (DRL) could lead to abnormal action selections of the agent, which may result in failure or even catastrophe in crucial decision processes. However, existing attacks only consider single-agent reinforcement learning (RL) systems, in which the only agent can observe the global state and have full control of the decision process. In this article, we explore a new backdoor attack paradigm in cooperative multi-agent reinforcement learning (CMARL) scenarios, where a group of agents coordinate with each other to achieve a common goal, while each agent can only observe the local state. In the proposed MARNet attack framework, we carefully design a pipeline of trigger design, action poisoning, and reward hacking modules to accommodate the cooperative multi-agent settings. In particular, as only a subset of agents can observe the triggers in their local observations, we maneuver their actions to the worst actions suggested by an expert policy model. Since the global reward in CMARL is aggregated by individual rewards from all agents, we propose to modify the reward in a way that boosts the bad actions of poisoned agents (agents who observe the triggers) but mitigates the influence on non-poisoned agents. We conduct extensive experiments on three classical CMARL algorithms VDN, COMA, and QMIX, in two popular CMARL games Predator Prey and SMAC. The results show that the baselines extended from single-agent DRL backdoor attacks seldom work in CMARL problems while MARNet performs well by reducing the utility under attack by nearly 100%. We apply fine-tuning as a potential defense against MARNet and demonstrate that fine-tuning cannot entirely eliminate the effect of the attack.

Discover the Hidden Attack Path in Multi-domain Cyberspace Based on Reinforcement Learning

Discover the Hidden Attack Path in Multiple Domain Cyberspace Based on Reinforcement Learning

Weakness Analysis of Cyberspace Configuration Based on Reinforcement Learning

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

Multiple Domain Cyberspace Attack and Defense Game Based on Reward Randomization Reinforcement Learning

Discovering Command and Control Channels Using Reinforcement Learning

Exposing Surveillance Detection Routes via Reinforcement Learning, Attack Graphs, and Cyber Terrain

Crown Jewels Analysis using Reinforcement Learning with Attack Graphs

Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learning

Applying Reinforcement Learning for Enhanced Cybersecurity against Adversarial Simulation

Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense

A method of network attack-defense game and collaborative defense decision-making based on hierarchical multi-agent reinforcement learning

A Hidden Attack Sequences Detection Method Based on Dynamic Reward Deep Deterministic Policy Gradient

Towards Secure Multi-Agent Deep Reinforcement Learning: Adversarial Attacks and Countermeasures

Reinforcement learning-based autonomous attacker to uncover computer network vulnerabilities

Discovering Command and Control (C2) Channels on Tor and Public Networks Using Reinforcement Learning

Applying Action Masking and Curriculum Learning Techniques to Improve Data Efficiency and Overall Performance in Operational Technology Cyber Security using Reinforcement Learning

Research and Challenges of Reinforcement Learning in Cyber Defense Decision-Making for Intranet Security

A Deep Reinforcement Learning-Based Method Applied for Solving Multi-Agent Defense and Attack Problems.

Reinforcement Learning Solution for Cyber-Physical Systems Security Against Replay Attacks

Deep Reinforcement Learning for Cyber Security