Abstract:Games, including abstract board games, constitute a convenient ground to create, design, and improve new AI methods. In this field, Monte Carlo Tree Search is a popular algorithm family, aiming to build game trees and explore them efficiently. Combinatorial Optimization, on the other hand, aims to model and solve problems with an objective to optimize and constraints to satisfy, and is less common in Game AI. We believe however that both methods can be combined efficiently, by injecting Combinatorial Optimization into Monte Carlo Tree Search to help the tree search, leading to a novel combination of these two techniques. Tested on the board game boop., our method beats 96% of the time the Monte Carlo Tree Search algorithm baseline. We conducted an ablation study to isolate and analyze which injections and combinations of injections lead to such performances. Finally, we opposed our AI method against human players on the Board Game Arena platform, and reached a 373 ELO rating after 51 boop. games, with a 69% win rate and finishing ranked 56th worldwide on the platform over 5,316 boop. players.
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is: **How to improve the performance of Monte Carlo Tree Search (MCTS) in board games by injecting Combinatorial Optimization techniques into MCTS**. Specifically, the author attempts to apply combinatorial optimization techniques to the three key steps of MCTS (selection, expansion, and simulation) to improve the performance of MCTS on resource - constrained devices and verify whether this method can significantly increase the winning rate of AI in games.
### Research Background and Motivation
1. **Combination of MCTS and Combinatorial Optimization**
- Monte Carlo Tree Search (MCTS) is an artificial intelligence algorithm widely used in board games and other fields, especially good at handling games with high branching factors.
- Combinatorial Optimization is committed to finding the optimal solution under certain constraint conditions, and is usually used to solve optimization problems of discrete variables.
- The author believes that combining these two methods can explore the game tree more effectively under limited computing resources, thereby improving the performance of AI.
2. **Research Motivation**
- Although existing AI methods (such as deep reinforcement learning) are powerful, they may require a large amount of computing resources in some cases.
- The author hopes that by introducing combinatorial optimization, good game performance can still be achieved when computing resources are limited.
- In addition, the author also hopes to push the knowledge boundaries in the AI field through this new combination method.
### Specific Objectives
- **Propose a new MCTS enhancement method**: Improve its decision - making process by introducing combinatorial optimization techniques at different stages of MCTS.
- **Verify the effectiveness of the new method**: Verify through experiments whether this method can significantly improve the performance of AI in actual games.
- **Analyze the effects of different combinations**: Analyze which injection methods of combinatorial optimization contribute the most to performance improvement through ablation study.
### Experimental Results
The author verified their hypothesis through a series of experiments:
- **Compared with basic MCTS**: In 100 games, MCTS - CO (MCTS with introduced combinatorial optimization) won 96% of the games, significantly outperforming basic MCTS.
- **Compared with heuristic methods**: Even using the same heuristic function, MCTS - CO is much stronger than AI that simply relies on heuristics, indicating that the performance improvement mainly comes from the combination of MCTS and combinatorial optimization, not just the role of the heuristic function.
- **Effects of different combinations**: By comparing the effects of different combinatorial optimization injection methods, it is found that combinatorial optimization in the selection, expansion, and simulation stages can all significantly improve performance, especially the optimization in the expansion stage is particularly effective when combined with other stages.
### Conclusion
This research shows that injecting combinatorial optimization techniques into MCTS can significantly improve the performance of AI in board games, especially when computing resources are limited. Through reasonable combinatorial optimization injection, AI can not only make better decisions in a short time, but also maintain a high winning rate in complex games.
### Formula Representation
The formulas involved in the paper mainly focus on the objective function and constraint conditions of the combinatorial optimization model. For example:
- **Objective Function**
\[
f(v_p, v_r, v_c)=\text{heuristics}(\text{game state after move }(v_p, v_r, v_c))
\]
Here, the heuristic function gives a score according to the change of the game state, and the score range is \([- 1,1]\).
- **Constraint Conditions**
- The selected piece must belong to the player's pool:
\[
\text{HasPiece}(v_p)=
\begin{cases}
\text{true}&\text{if }v_p\in\text{player pool}\\
\text{false}&\text{otherwise}
\end{cases}
\]