Abstract:Reinforcement learning agents are susceptible to evasion attacks during deployment. In single-agent environments, these attacks can occur through imperceptible perturbations injected into the inputs of the victim policy network. In multi-agent environments, an attacker can manipulate an adversarial opponent to influence the victim policy's observations indirectly. While adversarial policies offer a promising technique to craft such attacks, current methods are either sample-inefficient due to poor exploration strategies or require extra surrogate model training under the black-box assumption. To address these challenges, in this paper, we propose Intrinsically Motivated Adversarial Policy (IMAP) for efficient black-box adversarial policy learning in both single- and multi-agent environments. We formulate four types of adversarial intrinsic regularizers -- maximizing the adversarial state coverage, policy coverage, risk, or divergence -- to discover potential vulnerabilities of the victim policy in a principled way. We also present a novel bias-reduction method to balance the extrinsic objective and the adversarial intrinsic regularizers adaptively. Our experiments validate the effectiveness of the four types of adversarial intrinsic regularizers and the bias-reduction method in enhancing black-box adversarial policy learning across a variety of environments. Our IMAP successfully evades two types of defense methods, adversarial training and robust regularizer, decreasing the performance of the state-of-the-art robust WocaR-PPO agents by 34\%-54\% across four single-agent tasks. IMAP also achieves a state-of-the-art attacking success rate of 83.91\% in the multi-agent game YouShallNotPass. Our code is available at \url{<a class="link-external link-https" href="https://github.com/x-zheng16/IMAP" rel="external noopener nofollow">this https URL</a>}.

PALM: Preference-based Adversarial Manipulation against Deep Reinforcement Learning

MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning

STRAPPER: Preference-based Reinforcement Learning via Self-training Augmentation and Peer Regularization

Adversarial Policies: Attacking Deep Reinforcement Learning

Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations

Attacking and Defending Deep Reinforcement Learning Policies

Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses

Optimal Attack and Defense for Reinforcement Learning

Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy

Robust Deep Reinforcement Learning with Adversarial Attacks

Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Space

Online Robust Policy Learning in the Presence of Unknown Adversaries

Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies

Attacking Deep Reinforcement Learning with Decoupled Adversarial Policy

Robustifying Reinforcement Learning Agents via Action Space Adversarial Training

RL-Based Method for Benchmarking the Adversarial Resilience and Robustness of Deep Reinforcement Learning Policies

Adversarial Skill Learning for Robust Manipulation

PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation

Adversary Agnostic Robust Deep Reinforcement Learning

SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems

Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning