Monte Carlo Tree Search for a single target search game on a 2-D lattice

Elana Kozak,Scott Hottovy

DOI: https://doi.org/10.48550/arXiv.2011.14246

2020-11-29

Abstract:Monte Carlo Tree Search (MCTS) is a branch of stochastic modeling that utilizes decision trees for optimization, mostly applied to artificial intelligence (AI) game players. This project imagines a game in which an AI player searches for a stationary target within a 2-D lattice. We analyze its behavior with different target distributions and compare its efficiency to the Levy Flight Search, a model for animal foraging behavior. In addition to simulated data analysis we prove two theorems about the convergence of MCTS when computation constraints neglected.

Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to efficiently find a stationary target on a two - dimensional lattice (2 - D lattice) using the Monte Carlo Tree Search (MCTS) algorithm. Specifically, the researchers designed a "game" in which an AI player needs to find a target at a fixed position in a two - dimensional lattice with periodic boundaries.

### Research Background and Problem Description

1. **Search and Detection Problems**: Search and detection problems are widely used in many theoretical and practical scenarios. For example, in a simple search scenario, there may be a search agent and a target, which move randomly in space.

2. **Limitations of Existing Methods**: Although there are already many methods for finding these targets and have achieved varying degrees of success, the researchers proposed a new method - Monte Carlo Tree Search (MCTS) - in the hope of solving search and detection problems more effectively.

3. **Specific Problem Settings**:
- **Search Domain**: An \( N\times N \) two - dimensional lattice with periodic boundary conditions.
- **Target Distribution**: The target \( T \) is placed on the lattice according to a certain distribution \( P(x, y) \).
- **Initial Position**: The searcher starts from a preset position (usually (1, 1)).
- **Movement Rules**: The searcher can move one step in one of the four directions: up, down, left, or right each time.
- **Target Characteristics**: The target is stationary.
- **Optimal Path**: Defined as the path that enables the searcher to reach the target in the fewest number of steps.

### Research Objectives

The main objective of this paper is to evaluate the performance of MCTS in this specific search problem and compare it with other search strategies (such as Levy flight search and random walk). In addition, the research also proves two theorems regarding the convergence of MCTS:

1. **Convergence in the Case of Known Target**: When the position of the target is completely known (i.e., the target comes from the delta distribution), as the number of simulations increases, MCTS will converge to the optimal path.
2. **Convergence in the Case of Unknown Target**: When the position of the target is completely unknown (i.e., the target comes from the uniform distribution), as the size of the search grid increases, MCTS will converge to the Nearly Self - Avoiding Random Walk (NSARW).

Through these studies, the authors hope to demonstrate the effectiveness and superiority of MCTS in solving search problems under different information conditions.