Abstract:As a typical example of imperfect information game,Texas Hold'em poker has been an ongoing challenge in artificial intelligence for a long time.Especially in the case of multiplayer no-limit Texas Hold'em,a variety of factors should be considered,combined with its large solution space,making multiplayer no-limit Texas Hold'em AI more challenging.Generally,there are two approaches to handle this problem.The first is game-theoretic method,trying to search the game tree and find out the Nash Equilibrium points with a range of approaches.The second is termed knowledge-based method,learning human player's actions and betting patterns to provide AI with references and make corresponding decisions.In this paper,we propose a gaming policy in multiplayer no-limit Texas Hold'em based on hands prediction,which belongs to the latter category.The basic idea behind our approach is to imitate the process of "hands reading" taken by human beings.Hands reading is an essential skill in playing Texas Hold'em,which refers to predicting the rough probability distribution,or even the precise hands,of the opponents' cards.Hands reading is based on the actions taken by our opponents and the real-time information we perceived.As the development of gaming,players will take several action in each betting round (each player will normally take several actions in a single hand in total).The actions taken by players reveal the information of their hands,making hands reading possible.In this paper,we proposes a complete gaming framework from the perspective of imperfect information game.The applicable scope of this game framework is also discussed in the same section.Then we apply the proposed framework to Texas Hold'em.When designing our poker AI,we focus more on the prediction of the unknown information set.At each time point,our poker AI conducts Monte Carlo simulation to obtain the winning rate based on the current collected information and the predicted information.And winning rate is the most important factor our AI considers to make a decision.This paper presents a detailed description of the AI designation and corresponding implementation.We give the readers valuable references to implement their own multiplayer no-limit Texas Hold'em AI.To our best knowledge,this is the first paper that holistically designs and implements multiplayer (more than 3 players) no-limit Texas Hold'em AI based on hands prediction.Experiments are conducted to demonstrate the performance of our policy and the produced AI.In terms of predicting unknown information set,our approach increases the hands prediction accuracy by 6.65％ than Markov model.To intuitively illustrate the strength of our AI,we select 7 programs from 2015 Huawei Software Elite Challenge for comparison.Experiments are conducted under various conditions,including heads-up (2 players) and multiplayer no-limit Texas Hold'em.We select tournament as the competition rule,with one extra buy-in and a constant blind.Our program gets a winning rate of 89％ in heads-up poker,and places 1.74 in average in eight-player poker.Additionally,it gets the best performance in terms of lasting rounds,winning chips,etc.

AlphaDou: High-Performance End-to-End Doudizhu AI Integrating Bidding

Full DouZero+: Improving DouDizhu AI by Opponent Modeling, Coach-Guided Training and Bidding Learning

DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learning

DouRN: Improving DouZero by Residual Neural Networks

DanZero+: Dominating the GuanDan Game through Reinforcement Learning

RARSMSDou: Master the Game of DouDiZhu With Deep Reinforcement Learning Algorithms

DanZero: Mastering GuanDan Game with Reinforcement Learning

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner

Suphx: Mastering Mahjong with Deep Reinforcement Learning

PerfectDou: Dominating DouDizhu with Perfect Information Distillation

JP-DouZero: an enhanced DouDiZhu AI based on reinforcement learning with peasant collaboration and intrinsic rewards.

AlphaHoldem: High-Performance Artificial Intelligence for Heads-Up No-Limit Poker via End-to-End Reinforcement Learning

Combining Tree Search and Action Prediction for State-of-the-Art Performance in DouDiZhu

Combinatorial Q-Learning for Dou Di Zhu.

Game in Multiplayer No-Limit Texas Hold'Em Based on Hands Prediction

A Deep Reinforcement Learning-Based Approach in Porker Game

Research on Action Strategies and Simulations of DRL and MCTS-based Intelligent Round Game

A novel deep residual network-based incomplete information competition strategy for four-players Mahjong games

Towards a Competitive 3-Player Mahjong AI Using Deep Reinforcement Learning

MathDQN: Solving Arithmetic Word Problems Via Deep Reinforcement Learning.

Efficient Learning for AlphaZero via Path Consistency.