Abstract:This study probes the vulnerabilities of cooperative multi-agent reinforcement learning (c-MARL) under adversarial attacks, a critical determinant of c-MARL's worst-case performance prior to real-world implementation. Current observation-based attacks, constrained by white-box assumptions, overlook c-MARL's complex multi-agent interactions and cooperative objectives, resulting in impractical and limited attack capabilities. To address these shortcomes, we propose Adversarial Minority Influence (AMI), a practical and strong for c-MARL. AMI is a practical black-box attack and can be launched without knowing victim parameters. AMI is also strong by considering the complex multi-agent interaction and the cooperative goal of agents, enabling a single adversarial agent to unilaterally misleads majority victims to form targeted worst-case cooperation. This mirrors minority influence phenomena in social psychology. To achieve maximum deviation in victim policies under complex agent-wise interactions, our unilateral attack aims to characterize and maximize the impact of the adversary on the victims. This is achieved by adapting a unilateral agent-wise relation metric derived from mutual information, thereby mitigating the adverse effects of victim influence on the adversary. To lead the victims into a jointly detrimental scenario, our targeted attack deceives victims into a long-term, cooperatively harmful situation by guiding each victim towards a specific target, determined through a trial-and-error process executed by a reinforcement learning agent. Through AMI, we achieve the first successful attack against real-world robot swarms and effectively fool agents in simulated environments into collectively worst-case scenarios, including Starcraft II and Multi-agent Mujoco. The source code and demonstrations can be found at: <a class="link-external link-https" href="https://github.com/DIG-Beihang/AMI" rel="external noopener nofollow">this https URL</a>.

A Spatiotemporal Stealthy Backdoor Attack against Cooperative Multi-Agent Deep Reinforcement Learning

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systems

Backdoor Attack Influence Assessment and Analysis on Multi-Agent Training

Adversarial attacks on cooperative multi-agent deep reinforcement learning: a dynamic group-based adversarial example transferability method

Backdoors Stuck At The Frontdoor: Multi-Agent Backdoor Attacks That Backfire

A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learning

BadRL: Sparse Targeted Backdoor Attack Against Reinforcement Learning

Mitigating Deep Reinforcement Learning Backdoors in the Neural Activation Space

Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guarantee

Adversarial Attacks on Multiagent Deep Reinforcement Learning Models in Continuous Action Space

CuDA2: An approach for Incorporating Traitor Agents into Cooperative Multi-Agent Systems

Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack

Stealthy and efficient adversarial attacks against deep reinforcement learning

Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence

Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents

BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents

Design of intentional backdoors in sequential models

PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learning

Recover Triggered States: Protect Model Against Backdoor Attack in Reinforcement Learning

Escaping Backdoor Attack Detection of Deep Learning