Abstract:This paper studies decision-making in two-player scenarios where the type (e.g. adversary, neutral, or teammate) of the other agent (opponent) is uncertain to the decision-making agent (protagonist), which is an abstraction of security-domain applications. In these settings, the reward for the protagonist agent depends on the type of the opponent, but this is private information known only to the opponent itself, and thus hidden from the protagonist. In contrast, as is often the case, the type of the protagonist agent is assumed to be known to the opponent, and this information-asymmetry significantly complicates the protagonist's decision-making. In particular, to determine the best actions to take, the protagonist agent must infer the opponent type from the observations and agent modeling. To address this problem, this paper presents an opponent-type deduction module based on Bayes' rule. This inference module takes as input the imagined opponent's decision-making rule (opponent model) as well as the observed opponent's history of actions and states, and outputs a belief over the opponent's hidden type. A multiagent reinforcement learning approach is used to develop this game-theoretic opponent model through self-play, which avoids the expensive data collection step that requires interaction with a real opponent. Besides, this multiagent approach also captures the strategy interaction and reasoning between agents. In addition, we apply ensemble training to avoid over-fitting to a single opponent model during the training. As a result, the learned protagonist policy is also effective against unseen opponents. Experimental results show that the proposed game-theoretic modeling, explicit opponent type inference and the ensemble training significantly improves the decision-making performance over baseline approaches, and generalizes well against adversaries that have not been seen during the training.

Opponent Modeling in Adversarial Multi-robot System Simulation

A Plan Recognition Approach for Agent in Adversarial Multi-Agent System

Model-Based Opponent Modeling

Towards a cooperative soccer team in RoboCup

Modeling Opponent Actions for Table-Tennis Playing Robot.

Opponent Modeling in Deep Reinforcement Learning

Opponent learning for multi-agent system simulation

Balancing Safety and Exploitability in Opponent Modeling.

Modeling and Control Architecture for the Competitive Networked Robot System Based on POMDP

Opponent Modeling in Poker Games

Opponent Modeling in Multiplayer Imperfect-Information Games

Design of Strategy Based on Home and Opponent Information in Robot Soccer Competition

Opponent Modeling with Trajectory Representation Clustering

Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning

Studying of feature-based match modeling in robot soccer and decision-making system's self-learning

A muti-agent defensive strategy based on monte carlo method

Detecting and Tracing Multi-Strategic Agents with Opponent Modelling and Bayesian Policy Reuse

Behavior Reasoning for Opponent Agents in Multi-Agent Learning Systems

Multi-robot adversarial system based on multi-agent theory

Multi-Mobile Robot's Oppositional Strategy System Based on Discrete Variable Action Space

Autonomous intelligent decision-making system based on Bayesian SOM neural network for robot soccer