Abstract:Multi-Agent Reinforcement Learning (MARL) is vulnerable to Adversarial Machine Learning (AML) attacks and needs adequate defences before it can be used in real world applications. We have conducted a survey into the use of execution-time AML attacks against MARL and the defences against those attacks. We surveyed related work in the application of AML in Deep Reinforcement Learning (DRL) and Multi-Agent Learning (MAL) to inform our analysis of AML for MARL. We propose a novel perspective to understand the manner of perpetrating an AML attack, by defining Attack Vectors. We develop two new frameworks to address a gap in current modelling frameworks, focusing on the means and tempo of an AML attack against MARL, and identify knowledge gaps and future avenues of research.

What problem does this paper attempt to address?

This paper attempts to address the issues of adversarial machine learning (AML) attacks and defenses in multi - agent reinforcement learning (MARL). Specifically, the paper focuses on the fact that in practical applications, MARL systems are vulnerable to AML attacks, and thus effective defense measures are required to ensure their security and reliability. The main contributions of the paper include: 1. **Investigating existing AML attack and defense methods**: The paper conducts an extensive literature review of AML attacks and their defenses at execution time, covering deep reinforcement learning (DRL), multi - agent learning (MAL), and the broader MARL field. 2. **Proposing a new attack perspective - Attack Vectors**: In order to better understand how to carry out AML attacks, the author defines a new concept of "Attack Vectors" to describe the means of attack, the knowledge used, and the attack targets. 3. **Developing two new frameworks**: These two frameworks aim to fill the gaps in existing modeling frameworks, focusing on describing the time, means, and intensity of AML attacks, especially in MARL applications. 4. **Identifying knowledge gaps and proposing future research directions**: Through systematic literature analysis, the author points out the deficiencies in current research and proposes future research directions to promote the further development of this field. ### Specific Problem Description The paper points out that although deep reinforcement learning (DRL) has made significant progress in the past decade, its application in multi - agent environments still faces a major challenge: vulnerability to adversarial machine learning (AML) attacks. These attacks can cause neural networks to produce significantly different outputs through tiny, human - imperceptible input changes, thus affecting the behavior of agents. In multi - agent environments, this vulnerability is particularly prominent because the interaction between multiple agents makes the complexity and potential impact of attacks greater. To address this challenge, the paper analyzes in detail the existing AML attack means and explores effective defense strategies against these attacks. In addition, the author also pays special attention to the communication mechanisms in multi - agent systems and proposes a classification of attack vectors for communication perturbations, providing new perspectives and tools for future defense research. ### Conclusion In conclusion, this paper aims to reveal the vulnerability of multi - agent reinforcement learning systems under adversarial attacks through systematic literature review and innovative methodologies, and to provide theoretical basis and technical support for building more robust MARL systems.

SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

Adversarial attacks in consensus-based multi-agent reinforcement learning

Adversarial Markov Games: On Adaptive Decision-Based Attacks and Defenses

Camouflage Adversarial Attacks on Multiple Agent Systems

Robust Communicative Multi-Agent Reinforcement Learning with Active Defense

Adversarial Machine Learning: Bayesian Perspectives

Defenses in Adversarial Machine Learning: A Survey

Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective

Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influence

On the Robustness of Cooperative Multi-Agent Reinforcement Learning

You Don't Need Robust Machine Learning to Manage Adversarial Attack Risks

Adversarial Machine Learning Attacks against Intrusion Detection Systems: A Survey on Strategies and Defense

SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems

A Survey on Adversarial Machine Learning for Code Data: Realistic Threats, Countermeasures, and Interpretations

Attacking c-MARL More Effectively: A Data Driven Approach

What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?

Grey-box Adversarial Attack on Communication in Multi-agent Reinforcement Learning

Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain

Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning