Abstract:Regret minimization is a powerful tool for solving large-scale extensive-form games. State-of-the-art methods rely on minimizing regret locally at each decision point. In this work we derive a new framework for regret minimization on sequential decision problems and extensive-form games with general compact convex sets at each decision point and general convex losses, as opposed to prior work which has been for simplex decision points and linear losses. We call our framework laminar regret decomposition. It generalizes the CFR algorithm to this more general setting. Furthermore, our framework enables a new proof of CFR even in the known setting, which is derived from a perspective of decomposing polytope regret, thereby leading to an arguably simpler interpretation of the algorithm. Our generalization to convex compact sets and convex losses allows us to develop new algorithms for several problems: regularized sequential decision making, regularized Nash equilibria in extensive-form games, and computing approximate extensive-form perfect equilibria. Our generalization also leads to the first regret-minimization algorithm for computing reduced-normal-form quantal response equilibria based on minimizing local regrets. Experiments show that our framework leads to algorithms that scale at a rate comparable to the fastest variants of counterfactual regret minimization for computing Nash equilibrium, and therefore our approach leads to the first algorithm for computing quantal response equilibria in extremely large games. Finally we show that our framework enables a new kind of scalable opponent exploitation approach.

Imization for extensive games with imperfect information

Lazy-CFR: a Fast Regret Minimization Algorithm for Extensive Games with Imperfect Information.

No-Regret Learning in Extensive-Form Games with Imperfect Recall

Efficient CFR for Imperfect Information Games with Instant Updates

Deep Counterfactual Regret Minimization

RLCFR: Minimize counterfactual regret by deep reinforcement learning

Combining Counterfactual Regret Minimization with Information Gain to Solve Extensive Games with Unknown Environments

Modeling Other Players with Bayesian Beliefs for Games with Incomplete Information

Online Convex Optimization for Sequential Decision Processes and Extensive-Form Games

GPU-Accelerated Counterfactual Regret Minimization

Scalable sub-game solving for imperfect-information games

D2CFR: Minimize Counterfactual Regret With Deep Dueling Neural Network

Minimizing Weighted Counterfactual Regret with Optimistic Online Mirror Descent

Faster Game Solving via Hyperparameter Schedules

CFR-p: Counterfactual Regret Minimization with Hierarchical Policy Abstraction, and its Application to Two-player Mahjong

Parallel Counterfactual Regret Minimization in Crowdsourcing Imperfect-information Expanded Game

Model-Free Neural Counterfactual Regret Minimization with Bootstrap Learning

RM-FSP: Regret Minimization Optimizes Neural Fictitious Self-Play

A Survey of Nash Equilibrium Strategy Solving Based on CFR

Near-Optimal $Φ$-Regret Learning in Extensive-Form Games

A Fast-Convergence Method of Monte Carlo Counterfactual Regret Minimization for Imperfect Information Dynamic Games