Abstract:To learn good joint policies for multi-agent collaboration with imperfect information remains a fundamental challenge. While for two-player zero-sum games, coordinate-ascent approaches (optimizing one agent's policy at a time, e.g., self-play) work with guarantees, in multi-agent cooperative setting they often converge to sub-optimal Nash equilibrium. On the other hand, directly modeling joint policy changes in imperfect information game is nontrivial due to complicated interplay of policies (e.g., upstream updates affect downstream state reachability). In this paper, we show global changes of game values can be decomposed to policy changes localized at each information set, with a novel term named policy-change density. Based on this, we propose Joint Policy Search(JPS) that iteratively improves joint policies of collaborative agents in imperfect information games, without re-evaluating the entire game. On multi-agent collaborative tabular games, JPS is proven to never worsen performance and can improve solutions provided by unilateral approaches (e.g, CFR), outperforming algorithms designed for collaborative policy learning (e.g. BAD). Furthermore, for real-world games, JPS has an online form that naturally links with gradient updates. We test it to Contract Bridge, a 4-player imperfect-information game where a team of $2$ collaborates to compete against the other. In its bidding phase, players bid in turn to find a good contract through a limited information channel. Based on a strong baseline agent that bids competitive bridge purely through domain-agnostic self-play, JPS improves collaboration of team players and outperforms WBridge5, a championship-winning software, by $+0.63$ IMPs (International Matching Points) per board over 1k games, substantially better than previous SoTA ($+0.41$ IMPs/b) under Double-Dummy evaluation.

A Simple, Solid, and Reproducible Baseline for Bridge Bidding AI

Automatic Bridge Bidding Using Deep Reinforcement Learning

AI Enabled Bridge Bidding Supporting Interactive Visualization

Aibench: an industry standard ai benchmark suite

Competitive Bridge Bidding with Deep Neural Networks

AIBench: An Agile Domain-specific Benchmarking Methodology and an AI Benchmark Suite

AIBench: an Industry Standard AI Benchmark Suite from Internet Services.

Alternate inference-decision reinforcement learning with generative adversarial inferring for bridge bidding

AIBench Scenario: Scenario-distilling AI Benchmarking

OpenHoldem: A Benchmark for Large-Scale Imperfect-Information Game Research

Welfare Diplomacy: Benchmarking Language Model Cooperation

Joint Policy Search for Multi-agent Collaboration with Imperfect Information

StarAI: Reducing incompleteness in the game of Bridge using PLP

Full DouZero+: Improving DouDizhu AI by Opponent Modeling, Coach-Guided Training and Bidding Learning

Auction-Based Regulation for Artificial Intelligence

Towards Game-Playing AI Benchmarks via Performance Reporting Standards

Designing Skill-Compatible AI: Methodologies and Frameworks in Chess

Official International Mahjong: A New Playground for AI Research

Match Point AI: A Novel AI Framework for Evaluating Data-Driven Tennis Strategies

The CoachAI Badminton Environment: A Novel Reinforcement Learning Environment with Realistic Opponents (Student Abstract)

The CoachAI Badminton Environment: Bridging the Gap between a Reinforcement Learning Environment and Real-World Badminton Games