Abstract:Multi-objective game (MOG) is a fundamental model for the decision-making problems in which each player must consider multi-dimensional payoffs that reflect different objectives. Typically, solving MOG involves refining the set of equilibrium strategies, which is also known as MOG strategy selection (MOGS). However, existing MOG algorithms only allow one metric for MOGS, which limits the application in real-world scenarios where the players may have different preferences over multiple metrics. In this paper, we first develop a preference-based MOGS framework to encompass multiple metrics with different preferences in MOGS. Based on the framework, we introduce the concept of comprehensive evaluation value (CEV) to evaluate the quality of a strategy set given the preference of each metric. Using CEV as a reward signal, we formulate the problem of finding the optimal strategy set as a Markov decision process, and use deep reinforcement learning to train a policy for MOG strategy selection given the metrics and the corresponding preferences. Specifically, we combine a rational strategy filtering procedure with a Transformer-based encoder–decoder policy network to refine the strategies given the preferences, and then we use a revised REINFORCE algorithm to train the policy network. Besides, we introduce variable beam search decoding to improve the quality of a rollout by keeping track of the most promising strategy sets and choosing the best one. We benchmark our algorithm on the MOG instances generated by GAMUT, and extensive experiments demonstrate that our algorithm can generate the strategy set significantly better than the state-of-the-art baselines with lower computational overhead given different preferences. Furthermore, we compare our approach on real-world problems, showing the great advantages in both performance and runtime.

OB-HPPO: an Option and Intrinsic Curiosity Based Hierarchical Reinforcement Learning Approach for Real-Time Strategy Games

Hierarchical Reinforcement Learning for Multi-agent MOBA Game

A Curriculum Learning Based Multi-agent Reinforcement Learning Method for Realtime Strategy Game

Multi-Robot Real-time Game Strategy Learning Based on Deep Reinforcement Learning.

Hierarchical Deep Reinforcement Learning Agent with Counter Self-play on Competitive Games

Hierarchical reinforcement learning with unlimited option scheduling for sparse rewards in continuous spaces

Dynamic Hierarchies in Hierarchical Reinforcement Learning

Hierarchical Macro Strategy Model for MOBA Game AI

Exploring Dominant Strategies in Iterated and Evolutionary Games: a Multi-Agent Reinforcement Learning Approach

AIBPO: Combine the Intrinsic Reward and Auxiliary Task for 3D Strategy Game

Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning

A new multi-agent reinforcement learning approach

Hierarchical Reinforcement Learning with OMQ

Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization

Intrinsically Motivated Hierarchical Policy Learning in Multi-objective Markov Decision Processes

Optimistic Exploration Based on Categorical-DQN for Cooperative Markov Games.

Option-based Multi-agent Exploration

Hybridization of evolutionary algorithm and deep reinforcement learning for multi-objective orienteering optimization

Deep reinforcement learning for multi-objective game strategy selection

POAQL: A Partially Observable Altruistic Q-Learning Method for Cooperative Multi-Agent Reinforcement Learning

Two Heads Are Better Than One: A Simple Exploration Framework for Efficient Multi-Agent Reinforcement Learning.