Abstract:In two-player zero-sum games, if both players minimize their average external regret, then the average of the strategy profiles converges to a Nash equilibrium. For n-player general-sum games, however, theoretical guarantees for regret minimization are less understood. Nonetheless, Counterfactual Regret Minimization (CFR), a popular regret minimization algorithm for extensive-form games, has generated winning three-player Texas Hold'em agents in the Annual Computer Poker Competition (ACPC). In this paper, we provide the first set of theoretical properties for regret minimization algorithms in non-zero-sum games by proving that solutions eliminate iterative strict domination. We formally define \emph{dominated actions} in extensive-form games, show that CFR avoids iteratively strictly dominated actions and strategies, and demonstrate that removing iteratively dominated actions is enough to win a mock tournament in a small poker game. In addition, for two-player non-zero-sum games, we bound the worst case performance and show that in practice, regret minimization can yield strategies very close to equilibrium. Our theoretical advancements lead us to a new modification of CFR for games with more than two players that is more efficient and may be used to generate stronger strategies than previously possible. Furthermore, we present a new three-player Texas Hold'em poker agent that was built using CFR and a novel game decomposition method. Our new agent wins the three-player events of the 2012 ACPC and defeats the winning three-player programs from previous competitions while requiring less resources to generate than the 2011 winner. Finally, we show that our CFR modification computes a strategy of equal quality to our new agent in a quarter of the time of standard CFR using half the memory.

Monte Carlo Continual Resolving for Online Strategy Computation in Imperfect Information Games

A Fast-Convergence Method of Monte Carlo Counterfactual Regret Minimization for Imperfect Information Dynamic Games

Perfect Information Monte Carlo with Postponing Reasoning

Continual Depth-limited Responses for Computing Counter-strategies in Sequential Games

Monte Carlo Neural Fictitious Self-Play: Achieve Approximate Nash equilibrium of Imperfect-Information Games.

Accelerating Nash Equilibrium Convergence in Monte Carlo Settings Through Counterfactual Value Based Fictitious Play

Efficient CFR for Imperfect Information Games with Instant Updates

Regret Minimization in Non-Zero-Sum Games with Applications to Building Champion Multiplayer Computer Poker Agents

Scalable sub-game solving for imperfect-information games

Monte Carlo Game Solver

Sound Algorithms in Imperfect Information Games

Monte Carlo Neural Fictitious Self-Play: Approach to Approximate Nash equilibrium of Imperfect-Information Games

Online Convex Optimization for Sequential Decision Processes and Extensive-Form Games

On self-play computation of equilibrium in poker

Solving Imperfect Information Poker Games Using Monte Carlo Search and POMDP Models

Simultaneous incremental support adjustment and metagame solving: An equilibrium-finding framework for continuous-action games

GPU-Accelerated Counterfactual Regret Minimization

Finding and Certifying (Near-)Optimal Strategies in Black-Box Extensive-Form Games

The Computational Complexity of Single-Player Imperfect-Recall Games

Deep Counterfactual Regret Minimization

A Monte Carlo Neural Fictitious Self-Play approach to approximate Nash Equilibrium in imperfect-information dynamic games