Abstract:The ex ante equilibrium for two-team zero-sum games, where agents within each team collaborate to compete against the opposing team, is known to be the best a team can do for coordination. Many existing works on ex ante equilibrium solutions are aiming to extend the scope of ex ante equilibrium solving to large-scale team games based on Policy Space Response Oracle (PSRO). However, the joint team policy space constructed by the most prominent method, Team PSRO, cannot cover the entire team policy space in heterogeneous team games where teammates play distinct roles. Such insufficient policy expressiveness causes Team PSRO to be trapped into a sub-optimal ex ante equilibrium with significantly higher exploitability and never converges to the global ex ante equilibrium. To find the global ex ante equilibrium without introducing additional computational complexity, we first parameterize heterogeneous policies for teammates, and we prove that optimizing the heterogeneous teammates' policies sequentially can guarantee a monotonic improvement in team rewards. We further propose Heterogeneous-PSRO (H-PSRO), a novel framework for heterogeneous team games, which integrates the sequential correlation mechanism into the PSRO framework and serves as the first PSRO framework for heterogeneous team games. We prove that H-PSRO achieves lower exploitability than Team PSRO in heterogeneous team games. Empirically, H-PSRO achieves convergence in matrix heterogeneous games that are unsolvable by non-heterogeneous baselines. Further experiments reveal that H-PSRO outperforms non-heterogeneous baselines in both heterogeneous team games and homogeneous settings.

Heterogeneous Roles against Assignment Based Policies in Two vs Two Target Defense Game

Guarding a Target Area from a Heterogeneous Group of Cooperative Attackers

Analysis of Role Switch for Cooperative Target Defense Differential Game

Cooperative defense of a territorial-constrained target in a target-attacker-defender game

Solutions for Multiple-Defender Single-Attacker Convex Target Guarding Games: Verification via Parametric Optimization

Multi-Agent Reach-Avoid Games: Two Attackers Versus One Defender and Mixed Integer Programming

A Model for Multi-Agent Heterogeneous Interaction Problems

Guarding a Subspace in High-Dimensional Space With Two Defenders and One Attacker

Multi-dimensional Network Security Game: How do attacker and defender battle on parallel targets?

Optimal strategies design of active target defense differential game

Matching-Based Capture-the-Flag Games for Multi-Agent Systems

Synchronous intercept strategies for a robotic defense-intrusion game with two defenders

Computing Ex Ante Equilibrium in Heterogeneous Zero-Sum Team Games

NMPC-Based Cooperative Strategy For A Target Pair To Lure Two Attackers Into Collision

Receding Horizon Defense Strategy for Reach-Avoid Games with Uncertainties Via Pairwise Outcomes

Smooth Information Gathering in Two-Player Noncooperative Games

Reach-Avoid Games with Two Defenders and One Attacker: an Analytical Approach

Attack versus defense: A strategic rationale for role differentiation in conflict

Adaptive Attacker Strategy Development Against Moving Target Cyber Defenses

Task Assignment for Multiplayer Reach-Avoid Games in Convex Domains via Analytical Barriers

Dynamic Defender-Attacker Blotto Game