Abstract:Reinforcement learning agents are susceptible to evasion attacks during deployment. In single-agent environments, these attacks can occur through imperceptible perturbations injected into the inputs of the victim policy network. In multi-agent environments, an attacker can manipulate an adversarial opponent to influence the victim policy's observations indirectly. While adversarial policies offer a promising technique to craft such attacks, current methods are either sample-inefficient due to poor exploration strategies or require extra surrogate model training under the black-box assumption. To address these challenges, in this paper, we propose Intrinsically Motivated Adversarial Policy (IMAP) for efficient black-box adversarial policy learning in both single- and multi-agent environments. We formulate four types of adversarial intrinsic regularizers -- maximizing the adversarial state coverage, policy coverage, risk, or divergence -- to discover potential vulnerabilities of the victim policy in a principled way. We also present a novel bias-reduction method to balance the extrinsic objective and the adversarial intrinsic regularizers adaptively. Our experiments validate the effectiveness of the four types of adversarial intrinsic regularizers and the bias-reduction method in enhancing black-box adversarial policy learning across a variety of environments. Our IMAP successfully evades two types of defense methods, adversarial training and robust regularizer, decreasing the performance of the state-of-the-art robust WocaR-PPO agents by 34\%-54\% across four single-agent tasks. IMAP also achieves a state-of-the-art attacking success rate of 83.91\% in the multi-agent game YouShallNotPass. Our code is available at \url{<a class="link-external link-https" href="https://github.com/x-zheng16/IMAP" rel="external noopener nofollow">this https URL</a>}.

Robust Multi-Agent Reinforcement Learning against Adversaries on Observation

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning

Adversarial Attacks on Multi-Agent Communication

Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy

Robustifying Reinforcement Learning Agents via Action Space Adversarial Training

Multiple-Model Based Defense for Deep Reinforcement Learning Against Adversarial Attack

Towards Secure Multi-Agent Deep Reinforcement Learning: Adversarial Attacks and Countermeasures

SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems

On the Robustness of Cooperative Multi-Agent Reinforcement Learning

Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations

Adversarial Policies: Attacking Deep Reinforcement Learning

Camouflage Adversarial Attacks on Multiple Agent Systems

Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers

Adversarial attacks on cooperative multi-agent deep reinforcement learning: a dynamic group-based adversarial example transferability method

Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation.

Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Space

Robust Deep Reinforcement Learning with Adversarial Attacks

Deep-Attack over the Deep Reinforcement Learning

Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning

Adversarial Attacks on Reinforcement Learning Agents for Command and Control