Artificial intelligence for Bidding Hex

Sam Payne,Elina Robeva

DOI: https://doi.org/10.48550/arXiv.0812.3677

2008-12-18

Combinatorics

Abstract:We present a Monte Carlo algorithm for efficiently finding near optimal moves and bids in the game of Bidding Hex. The algorithm is based on the recent solution of Random-Turn Hex by Peres, Schramm, Sheffield, and Wilson together with Richman's work connecting random-turn games to bidding games.

What problem does this paper attempt to address?

Pheromone-Based pre-processing to improve move generation in board games

Shenyi Chen,Hui Qian,Miaoliang Zhu

DOI: https://doi.org/10.1007/11736639_84

2006-01-01

Abstract:We propose a method to improve generating moves in board games. The basic idea comes from swarm intelligence. The board game program runs Pheromone-based Pre-processing algorithm (PP) to construct two heuristic game trees apart from the game search tree, and learns from winners' moves by laying pheromone trails on heuristic game trees before game starts. Then the program picks moves on the basis of their pheromone quantity to start game tree search when playing games. The results of our experiments show that the method is efficient and promising.
AlphaDou: High-Performance End-to-End Doudizhu AI Integrating Bidding

Chang Lei,Huan Lei

2024-09-13

Abstract:Artificial intelligence for card games has long been a popular topic in AI research. In recent years, complex card games like Mahjong and Texas Hold'em have been solved, with corresponding AI programs reaching the level of human experts. However, the game of Doudizhu presents significant challenges due to its vast state/action space and unique characteristics involving reasoning about competition and cooperation, making the game extremely difficult to solve.The RL model Douzero, trained using the Deep Monte Carlo algorithm framework, has shown excellent performance in Doudizhu. However, there are differences between its simplified game environment and the actual Doudizhu environment, and its performance is still a considerable distance from that of human experts. This paper modifies the Deep Monte Carlo algorithm framework by using reinforcement learning to obtain a neural network that simultaneously estimates win rates and expectations. The action space is pruned using expectations, and strategies are generated based on win rates. The modified algorithm enables the AI to perform the full range of tasks in the Doudizhu game, including bidding and cardplay. The model was trained in a actual Doudizhu environment and achieved state-of-the-art performance among publicly available models. We hope that this new framework will provide valuable insights for AI development in other bidding-based games.

Artificial Intelligence,Computer Science and Game Theory,Multiagent Systems
Discrete bidding games

Mike Develin,Sam Payne

DOI: https://doi.org/10.48550/arXiv.0801.0579

2010-07-10

Abstract:We study variations on combinatorial games in which, instead of alternating moves, the players bid with discrete bidding chips for the right to determine who moves next. We consider both symmetric and partisan games, and explore differences between discrete bidding games and Richman games, which allow real-valued bidding. Unlike Richman games, discrete bidding game variations of many familiar games, such as chess, Connect Four, and even Tic-Tac-Toe, are suitable for recreational play. We also present an analysis of Tic-Tac-Toe for both discrete and real-valued bidding.

Combinatorics
Monte Carlo Neural Fictitious Self-Play: Achieve Approximate Nash equilibrium of Imperfect-Information Games.

Li Zhang,Wei Wang,Shijian Li,Gang Pan

2019-01-01

Abstract:Researchers on artificial intelligence have achieved human-level intelligence in large-scale perfect-information games, but it is still a challenge to achieve (nearly) optimal results (in other words, an approximate Nash Equilibrium) in large-scale imperfect-information games (i.e. war games, football coach or business strategies). Neural Fictitious Self Play (NFSP) is an effective algorithm for learning approximate Nash equilibrium of imperfect-information games from self-play without prior domain knowledge. However, it relies on Deep Q-Network, which is off-line and is hard to converge in online games with changing opponent strategy, so it canu0027t approach approximate Nash equilibrium in games with large search scale and deep search depth. In this paper, we propose Monte Carlo Neural Fictitious Self Play (MC-NFSP), an algorithm combines Monte Carlo tree search with NFSP, which greatly improves the performance on large-scale zero-sum imperfect-information games. Experimentally, we demonstrate that the proposed Monte Carlo Neural Fictitious Self Play can converge to approximate Nash equilibrium in games with large-scale search depth while the Neural Fictitious Self Play canu0027t. Furthermore, we develop Asynchronous Neural Fictitious Self Play (ANFSP). It use asynchronous and parallel architecture to collect game experience. In experiments, we show that parallel actor-learners have a further accelerated and stabilizing effect on training.
Bidding chess

Jay Bhat,Sam Payne

DOI: https://doi.org/10.48550/arXiv.0903.2995

2009-03-18

Abstract:An expository introduction to bidding chess and other bidding games. To appear in Mathematical Intelligencer.

History and Overview
Logic-based AI for Interpretable Board Game Winner Prediction with Tsetlin Machine

Charul Giri,Ole-Christoffer Granmo,Herke van Hoof,Christian D. Blakely

DOI: https://doi.org/10.48550/arXiv.2203.04378

2022-03-09

Abstract:Hex is a turn-based two-player connection game with a high branching factor, making the game arbitrarily complex with increasing board sizes. As such, top-performing algorithms for playing Hex rely on accurate evaluation of board positions using neural networks. However, the limited interpretability of neural networks is problematic when the user wants to understand the reasoning behind the predictions made. In this paper, we propose to use propositional logic expressions to describe winning and losing board game positions, facilitating precise visual interpretation. We employ a Tsetlin Machine (TM) to learn these expressions from previously played games, describing where pieces must be located or not located for a board position to be strong. Extensive experiments on $6\times6$ boards compare our TM-based solution with popular machine learning algorithms like XGBoost, InterpretML, decision trees, and neural networks, considering various board configurations with $2$ to $22$ moves played. On average, the TM testing accuracy is $92.1\%$, outperforming all the other evaluated algorithms. We further demonstrate the global interpretation of the logical expressions and map them down to particular board game configurations to investigate local interpretability. We believe the resulting interpretability establishes building blocks for accurate assistive AI and human-AI collaboration, also for more complex prediction tasks.

Artificial Intelligence,Machine Learning
Perfect Information Monte Carlo with Postponing Reasoning

Jérôme Arjonilla,Abdallah Saffidine,Tristan Cazenave

2024-08-05

Abstract:Imperfect information games, such as Bridge and Skat, present challenges due to state-space explosion and hidden information, posing formidable obstacles for search algorithms. Determinization-based algorithms offer a resolution by sampling hidden information and solving the game in a perfect information setting, facilitating rapid and effective action estimation. However, transitioning to perfect information introduces challenges, notably one called strategy fusion.This research introduces `Extended Perfect Information Monte Carlo' (EPIMC), an online algorithm inspired by the state-of-the-art determinization-based approach Perfect Information Monte Carlo (PIMC). EPIMC enhances the capabilities of PIMC by postponing the perfect information resolution, reducing alleviating issues related to strategy fusion. However, the decision to postpone the leaf evaluator introduces novel considerations, such as the interplay between prior levels of reasoning and the newly deferred resolution. In our empirical analysis, we investigate the performance of EPIMC across a range of games, with a particular focus on those characterized by varying degrees of strategy fusion. Our results demonstrate notable performance enhancements, particularly in games where strategy fusion significantly impacts gameplay. Furthermore, our research contributes to the theoretical foundation of determinization-based algorithms addressing challenges associated with strategy fusion.%, thereby enhancing our understanding of these algorithms within the context of imperfect information game scenarios.

Artificial Intelligence
An Information Theoretic Analysis of Decision in Computer Chess

Alexandru Godescu

DOI: https://doi.org/10.48550/arXiv.1112.2144

2011-12-10

Abstract:The basis of the method proposed in this article is the idea that information is one of the most important factors in strategic decisions, including decisions in computer chess and other strategy games. The model proposed in this article and the algorithm described are based on the idea of a information theoretic basis of decision in strategy games . The model generalizes and provides a mathematical justification for one of the most popular search algorithms used in leading computer chess programs, the fractional ply scheme. However, despite its success in leading computer chess applications, until now few has been published about this method. The article creates a fundamental basis for this method in the axioms of information theory, then derives the principles used in programming the search and describes mathematically the form of the coefficients. One of the most important parameters of the fractional ply search is derived from fundamental principles. Until now this coefficient has been usually handcrafted or determined from intuitive elements or data mining. There is a deep, information theoretical justification for such a parameter. In one way the method proposed is a generalization of previous methods. More important, it shows why the fractional depth ply scheme is so powerful. It is because the algorithm navigates along the lines where the highest information gain is possible. A working and original implementation has been written and tested for this algorithm and is provided in the appendix. The article is essentially self-contained and gives proper background knowledge and references. The assumptions are intuitive and in the direction expected and described intuitively by great champions of chess.

Artificial Intelligence,Information Theory
On the Power of Refined Skat Selection

Stefan Edelkamp

DOI: https://doi.org/10.48550/arXiv.2104.02997

2021-04-07

Abstract:Skat is a fascinating combinatorial card game, show-casing many of the intrinsic challenges for modern AI systems such as cooperative and adversarial behaviors (among the players), randomness (in the deal), and partial knowledge (due to hidden cards). Given the larger number of tricks and higher degree of uncertainty, reinforcement learning is less effective compared to classical board games like Chess and Go. As within the game of Bridge, in Skat we have a bidding and trick-taking stage. Prior to the trick-taking and as part of the bidding process, one phase in the game is to select two skat cards, whose quality may influence subsequent playing performance drastically. This paper looks into different skat selection strategies. Besides predicting the probability of winning and other hand strength functions we propose hard expert-rules and a scoring functions based on refined skat evaluation features. Experiments emphasize the impact of the refined skat putting algorithm on the playing performance of the bots, especially for AI bidding and AI game selection.

Artificial Intelligence
Expected Work Search: Combining Win Rate and Proof Size Estimation

Owen Randall,Martin Müller,Ting Han Wei,Ryan Hayward

2024-05-09

Abstract:We propose Expected Work Search (EWS), a new game solving algorithm. EWS combines win rate estimation, as used in Monte Carlo Tree Search, with proof size estimation, as used in Proof Number Search. The search efficiency of EWS stems from minimizing a novel notion of Expected Work, which predicts the expected computation required to solve a position. EWS outperforms traditional solving algorithms on the games of Go and Hex. For Go, we present the first solution to the empty 5x5 board with the commonly used positional superko ruleset. For Hex, our algorithm solves the empty 8x8 board in under 4 minutes. Experiments show that EWS succeeds both with and without extensive domain-specific knowledge.

Artificial Intelligence
Competing in a Complex Hidden Role Game with Information Set Monte Carlo Tree Search

Jack Reinhardt

DOI: https://doi.org/10.48550/arXiv.2005.07156

2020-05-15

Abstract:Advances in intelligent game playing agents have led to successes in perfect information games like Go and imperfect information games like Poker. The Information Set Monte Carlo Tree Search (ISMCTS) family of algorithms outperforms previous algorithms using Monte Carlo methods in imperfect information games. In this paper, Single Observer Information Set Monte Carlo Tree Search (SO-ISMCTS) is applied to Secret Hitler, a popular social deduction board game that combines traditional hidden role mechanics with the randomness of a card deck. This combination leads to a more complex information model than the hidden role and card deck mechanics alone. It is shown in 10108 simulated games that SO-ISMCTS plays as well as simpler rule based agents, and demonstrates the potential of ISMCTS algorithms in complicated information set domains.

Artificial Intelligence,Multiagent Systems
Automatic Bridge Bidding Using Deep Reinforcement Learning

Chih-Kuan Yeh,Hsuan-Tien Lin

DOI: https://doi.org/10.48550/arXiv.1607.03290

2016-07-12

Abstract:Bridge is among the zero-sum games for which artificial intelligence has not yet outperformed expert human players. The main difficulty lies in the bidding phase of bridge, which requires cooperative decision making under partial information. Existing artificial intelligence systems for bridge bidding rely on and are thus restricted by human-designed bidding systems or features. In this work, we propose a pioneering bridge bidding system without the aid of human domain knowledge. The system is based on a novel deep reinforcement learning model, which extracts sophisticated features and learns to bid automatically based on raw card data. The model includes an upper-confidence-bound algorithm and additional techniques to achieve a balance between exploration and exploitation. Our experiments validate the promising performance of our proposed model. In particular, the model advances from having no knowledge about bidding to achieving superior performance when compared with a champion-winning computer bridge program that implements a human-designed bidding system.

Artificial Intelligence
Infinite Hex is arithmetic

Ilkka Törmä

2023-10-12

Abstract:Hex is a well known connection game in which two players attempt to connect opposite sides of the board by colored stones. In 2022, Hamkins and Leonessi introduced an infinite version, in which the goal is to construct a certain kind of two-way infinite path of adjacent stones. It was explicitly left open whether the winning condition is Borel. We prove that it is arithmetic, with complexity between $\Sigma^0_4$ and $\Delta^0_5$.

Logic,Combinatorics
Artificial Intelligence for Multi-Unit Auction design

Peyman Khezr,Kendall Taylor

2024-08-08

Abstract:Understanding bidding behavior in multi-unit auctions remains an ongoing challenge for researchers. Despite their widespread use, theoretical insights into the bidding behavior, revenue ranking, and efficiency of commonly used multi-unit auctions are limited. This paper utilizes artificial intelligence, specifically reinforcement learning, as a model free learning approach to simulate bidding in three prominent multi-unit auctions employed in practice. We introduce six algorithms that are suitable for learning and bidding in multi-unit auctions and compare them using an illustrative example. This paper underscores the significance of using artificial intelligence in auction design, particularly in enhancing the design of multi-unit auctions.

Computer Science and Game Theory,Artificial Intelligence,Theoretical Economics
Optimal Bidding Algorithms Against Cheating in Multiple-Object Auctions

Ming-Yang Kao,Junfeng Qi,Lei Tan

DOI: https://doi.org/10.48550/arXiv.cs/0011023

2000-11-17

Computational Engineering, Finance, and Science

Abstract:This paper studies some basic problems in a multiple-object auction model using methodologies from theoretical computer science. We are especially concerned with situations where an adversary bidder knows the bidding algorithms of all the other bidders. In the two-bidder case, we derive an optimal randomized bidding algorithm, by which the disadvantaged bidder can procure at least half of the auction objects despite the adversary's a priori knowledge of his algorithm. In the general $k$-bidder case, if the number of objects is a multiple of $k$, an optimal randomized bidding algorithm is found. If the $k-1$ disadvantaged bidders employ that same algorithm, each of them can obtain at least $1/k$ of the objects regardless of the bidding algorithm the adversary uses. These two algorithms are based on closed-form solutions to certain multivariate probability distributions. In situations where a closed-form solution cannot be obtained, we study a restricted class of bidding algorithms as an approximation to desired optimal algorithms.
Proof Number Based Monte-Carlo Tree Search

Jakub Kowalski,Elliot Doe,Mark H. M. Winands,Daniel Górski,Dennis J. N. J. Soemers

2024-05-29

Abstract:This paper proposes a new game-search algorithm, PN-MCTS, which combines Monte-Carlo Tree Search (MCTS) and Proof-Number Search (PNS). These two algorithms have been successfully applied for decision making in a range of domains. We define three areas where the additional knowledge provided by the proof and disproof numbers gathered in MCTS trees might be used: final move selection, solving subtrees, and the UCB1 selection mechanism. We test all possible combinations on different time settings, playing against vanilla UCT on several games: Lines of Action ($7$$\times$$7$ and $8$$\times$$8$ board sizes), MiniShogi, Knightthrough, and Awari. Furthermore, we extend this new algorithm to properly address games with draws, like Awari, by adding an additional layer of PNS on top of the MCTS tree. The experiments show that PN-MCTS is able to outperform MCTS in all tested game domains, achieving win rates up to 96.2% for Lines of Action.

Artificial Intelligence
Monte-Carlo Tree Search with Epsilon-Greedy for Game of Amazons

Chen Tian

DOI: https://doi.org/10.54254/2755-2721/6/20230956

2023-01-01

Abstract:The Game of the Amazons is an abstract strategy board game. It has a high computational complexity similar to the game of Go. Due to its NP-complete nature and large branching factor of game tree, finding the optimal move given a specific game state is infeasible and it is not trivial to design a computer algorithm that is competitive to an expert in the game of amazons. One way to tackle this problem is to leverage the Monte-Carlo Tree Search by using random simulations. In this article, a computationally cheap heuristic function is proposed and use together with Monte-Carlo Tree Search algorithm with Epsilon-Greedy policy aiming to design a competitive AI for the Game of the Amazon. The effectiveness of the -greedy based Monte-Carlo algorithm is compared to the widely used MCTS with Upper Confidence Bound and other classical tree search method such as breadth-first search, depth-first search, minmax search and alpha-beta pruning.
The Complexities of Random-Turn Hex, Square, and Triangle Games

Yi Yang,Shuigeng Zhou,Jihong Guan,Chuansheng Shen

DOI: https://doi.org/10.1109/tg.2020.3033720

IF: 1.237

2022-01-01

IEEE Transactions on Games

Abstract:In random-turn games, players toss a coin to decide who moves. This article studies the complexities of the algorithms for playing random-turn connection games perfectly on regular tessellations. Our study theoretically shows that there are algorithms playing random-turn hex, square, and triangle perfectly in O(n(9) . 2.618(n)), O(n(9) . 2.746(n)), andO(n(9) . 3.645(n)) time for each move, respectively, where n is the board size. We then implement the perfect-playing algorithm for the random-turn square andmeasure the actual running time it costs for eachmove. We then compute and analyze the game lengths on random-turn square, hex, and triangle and conjecture that the asymptotic complexity of their game lengths are the same. We finally compare the perfect-playing algorithm with the sampling algorithm by competing against each other, and the numbers of their wins and losses are reported.
Infinite-Duration Bidding Games

Guy Avni,Thomas A. Henzinger,Ventsislav Chonev

DOI: https://doi.org/10.48550/arXiv.1705.01433

2019-06-07

Abstract:Two-player games on graphs are widely studied in formal methods as they model the interaction between a system and its environment. The game is played by moving a token throughout a graph to produce an infinite path. There are several common modes to determine how the players move the token through the graph; e.g., in turn-based games the players alternate turns in moving the token. We study the {\em bidding} mode of moving the token, which, to the best of our knowledge, has never been studied in infinite-duration games. The following bidding rule was previously defined and called Richman bidding. Both players have separate {\em budgets}, which sum up to $1$. In each turn, a bidding takes place: Both players submit bids simultaneously, where a bid is legal if it does not exceed the available budget, and the higher bidder pays his bid to the other player and moves the token. The central question studied in bidding games is a necessary and sufficient initial budget for winning the game: a {\em threshold} budget in a vertex is a value $t \in [0,1]$ such that if Player $1$'s budget exceeds $t$, he can win the game, and if Player $2$'s budget exceeds $1-t$, he can win the game. Threshold budgets were previously shown to exist in every vertex of a reachability game, which have an interesting connection with {\em random-turn} games -- a sub-class of simple stochastic games in which the player who moves is chosen randomly. We show the existence of threshold budgets for a qualitative class of infinite-duration games, namely parity games, and a quantitative class, namely mean-payoff games. The key component of the proof is a quantitative solution to strongly-connected mean-payoff bidding games in which we extend the connection with random-turn games to these games, and construct explicit optimal strategies for both players.

Logic in Computer Science,Computer Science and Game Theory
Solving Six-Player Games Via Online Situation Estimation

Huale Li,Xuan Wang,Shuhan Qi,Yang Liu,Haojie Wang,Fengwei Jia,Jiajia Zhang

DOI: https://doi.org/10.1109/ICTAI.2019.00270

2019-01-01

Abstract:While the artificial intelligence theory for solving the perfect-information games has been well developed in recent years, great challenges are still posed in dealing with the imperfect-information game due to the huge state space and hidden information involved in it. In this paper, we design an online strategy solving framework for six-player no-limit Texas hold'em poker. Based on hand isomorphism and hand strength evalution, the framework provides an efficient situation estimation method for six-player poker. Such method could greatly reduce the the state space in six-player poker as well as effectively evaluate the current hands. The poker agent based on our method won the third place in the 2018 AAAI-ACPC.

Artificial intelligence for Bidding Hex

Pheromone-Based pre-processing to improve move generation in board games

AlphaDou: High-Performance End-to-End Doudizhu AI Integrating Bidding

Discrete bidding games

Monte Carlo Neural Fictitious Self-Play: Achieve Approximate Nash equilibrium of Imperfect-Information Games.

Bidding chess

Logic-based AI for Interpretable Board Game Winner Prediction with Tsetlin Machine

Perfect Information Monte Carlo with Postponing Reasoning

An Information Theoretic Analysis of Decision in Computer Chess

On the Power of Refined Skat Selection

Expected Work Search: Combining Win Rate and Proof Size Estimation

Competing in a Complex Hidden Role Game with Information Set Monte Carlo Tree Search

Automatic Bridge Bidding Using Deep Reinforcement Learning

Infinite Hex is arithmetic

Artificial Intelligence for Multi-Unit Auction design

Optimal Bidding Algorithms Against Cheating in Multiple-Object Auctions

Proof Number Based Monte-Carlo Tree Search

Monte-Carlo Tree Search with Epsilon-Greedy for Game of Amazons

The Complexities of Random-Turn Hex, Square, and Triangle Games

Infinite-Duration Bidding Games

Solving Six-Player Games Via Online Situation Estimation