Abstract:Recent works have revealed that backdoor attacks against Deep Reinforcement Learning (DRL) could lead to abnormal action selections of the agent, which may result in failure or even catastrophe in crucial decision processes. However, existing attacks only consider single-agent reinforcement learning (RL) systems, in which the only agent can observe the global state and have full control of the decision process. In this article, we explore a new backdoor attack paradigm in cooperative multi-agent reinforcement learning (CMARL) scenarios, where a group of agents coordinate with each other to achieve a common goal, while each agent can only observe the local state. In the proposed MARNet attack framework, we carefully design a pipeline of trigger design, action poisoning, and reward hacking modules to accommodate the cooperative multi-agent settings. In particular, as only a subset of agents can observe the triggers in their local observations, we maneuver their actions to the worst actions suggested by an expert policy model. Since the global reward in CMARL is aggregated by individual rewards from all agents, we propose to modify the reward in a way that boosts the bad actions of poisoned agents (agents who observe the triggers) but mitigates the influence on non-poisoned agents. We conduct extensive experiments on three classical CMARL algorithms VDN, COMA, and QMIX, in two popular CMARL games Predator Prey and SMAC. The results show that the baselines extended from single-agent DRL backdoor attacks seldom work in CMARL problems while MARNet performs well by reducing the utility under attack by nearly 100%. We apply fine-tuning as a potential defense against MARNet and demonstrate that fine-tuning cannot entirely eliminate the effect of the attack.

Strangeness-driven exploration in multi-agent reinforcement learning

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

Multiagent Reinforcement Learning for Strictly Constrained Tasks Based on Reward Recorder

Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?

S2rl

Episodic Multi-agent Reinforcement Learning with Curiosity-driven Exploration

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Multi-agent Exploration with Sub-state Entropy Estimation

Randomized Exploration in Cooperative Multi-Agent Reinforcement Learning

CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learning

Self-Motivated Multi-Agent Exploration

An off-policy multi-agent stochastic policy gradient algorithm for cooperative continuous control

Efficient Multi-Agent Exploration with Mutual-Guided Actor-Critic

SC-MAIRL: Semi-Centralized Multi-Agent Imitation Reinforcement Learning

MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search

Two Heads Are Better Than One: A Simple Exploration Framework for Efficient Multi-Agent Reinforcement Learning.

MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

Exploiting Semantic Epsilon Greedy Exploration Strategy in Multi-Agent Reinforcement Learning

Adaptive trajectory-constrained exploration strategy for deep reinforcement learning

Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing