Abstract:This study probes the vulnerabilities of cooperative multi-agent reinforcement learning (c-MARL) under adversarial attacks, a critical determinant of c-MARL's worst-case performance prior to real-world implementation. Current observation-based attacks, constrained by white-box assumptions, overlook c-MARL's complex multi-agent interactions and cooperative objectives, resulting in impractical and limited attack capabilities. To address these shortcomes, we propose Adversarial Minority Influence (AMI), a practical and strong for c-MARL. AMI is a practical black-box attack and can be launched without knowing victim parameters. AMI is also strong by considering the complex multi-agent interaction and the cooperative goal of agents, enabling a single adversarial agent to unilaterally misleads majority victims to form targeted worst-case cooperation. This mirrors minority influence phenomena in social psychology. To achieve maximum deviation in victim policies under complex agent-wise interactions, our unilateral attack aims to characterize and maximize the impact of the adversary on the victims. This is achieved by adapting a unilateral agent-wise relation metric derived from mutual information, thereby mitigating the adverse effects of victim influence on the adversary. To lead the victims into a jointly detrimental scenario, our targeted attack deceives victims into a long-term, cooperatively harmful situation by guiding each victim towards a specific target, determined through a trial-and-error process executed by a reinforcement learning agent. Through AMI, we achieve the first successful attack against real-world robot swarms and effectively fool agents in simulated environments into collectively worst-case scenarios, including Starcraft II and Multi-agent Mujoco. The source code and demonstrations can be found at: <a class="link-external link-https" href="https://github.com/DIG-Beihang/AMI" rel="external noopener nofollow">this https URL</a>.

Enhancing the robustness of QMIX against state-adversarial attacks

Enhancing the Robustness of QMIX against State-adversarial Attacks

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agents

On the Robustness of Cooperative Multi-Agent Reinforcement Learning

Attacking c-MARL More Effectively: A Data Driven Approach

Robust Multi-Agent Reinforcement Learning with State Uncertainty

What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?

Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning

Camouflage Adversarial Attacks on Multiple Agent Systems

Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence

MIR2: Towards Provably Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization

Soft-QMIX: Integrating Maximum Entropy For Monotonic Value Function Factorization

SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems

Sparse Adversarial Attack in Multi-agent Reinforcement Learning

Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization

Robust Multi-Agent Reinforcement Learning Driven by Correlated Equilibrium

Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers

Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning

Robust Communicative Multi-Agent Reinforcement Learning with Active Defense

Mutual Learning-Based Framework for Enhancing Robustness of Code Models Via Adversarial Training