Abstract:Reinforcement learning agents are susceptible to evasion attacks during deployment. In single-agent environments, these attacks can occur through imperceptible perturbations injected into the inputs of the victim policy network. In multi-agent environments, an attacker can manipulate an adversarial opponent to influence the victim policy's observations indirectly. While adversarial policies offer a promising technique to craft such attacks, current methods are either sample-inefficient due to poor exploration strategies or require extra surrogate model training under the black-box assumption. To address these challenges, in this paper, we propose Intrinsically Motivated Adversarial Policy (IMAP) for efficient black-box adversarial policy learning in both single- and multi-agent environments. We formulate four types of adversarial intrinsic regularizers -- maximizing the adversarial state coverage, policy coverage, risk, or divergence -- to discover potential vulnerabilities of the victim policy in a principled way. We also present a novel bias-reduction method to balance the extrinsic objective and the adversarial intrinsic regularizers adaptively. Our experiments validate the effectiveness of the four types of adversarial intrinsic regularizers and the bias-reduction method in enhancing black-box adversarial policy learning across a variety of environments. Our IMAP successfully evades two types of defense methods, adversarial training and robust regularizer, decreasing the performance of the state-of-the-art robust WocaR-PPO agents by 34\%-54\% across four single-agent tasks. IMAP also achieves a state-of-the-art attacking success rate of 83.91\% in the multi-agent game YouShallNotPass. Our code is available at \url{<a class="link-external link-https" href="https://github.com/x-zheng16/IMAP" rel="external noopener nofollow">this https URL</a>}.

RLUC: Strengthening Robustness by Attaching Constraint Considerations to Policy Network

Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies

Curiosity-Driven and Victim-Aware Adversarial Policies.

Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations

Robust Reinforcement Learning on State Observations with Learned Optimal Adversary

Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy

RL-Based Method for Benchmarking the Adversarial Resilience and Robustness of Deep Reinforcement Learning Policies

On the Robustness of Safe Reinforcement Learning under Observational Perturbations

On the Perturbed States for Transformed Input-robust Reinforcement Learning

Adversarial Policies: Attacking Deep Reinforcement Learning

Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses

Robust off-policy Reinforcement Learning via Soft Constrained Adversary

Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learning

Transferable Adversarial Attacks on Deep Reinforcement Learning with Domain Randomization

Belief-Enriched Pessimistic Q-Learning against Adversarial State Perturbations

Robust Multi-Agent Reinforcement Learning against Adversaries on Observation

Robust Deep Reinforcement Learning with Adversarial Attacks

Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms

Robust Deep Reinforcement Learning with Adaptive Adversarial Perturbations in Action Space

Adversary Agnostic Robust Deep Reinforcement Learning

Robustifying Reinforcement Learning Agents via Action Space Adversarial Training