Computational bounds for the 2048 game

Alexey Slizkov

DOI: https://doi.org/10.48550/arXiv.2303.07266

2023-03-09

Data Structures and Algorithms

Abstract:2048 is a single player video game, played by millions mostly on mobile devices. We prove rigorously for the first time that there is an algorithm with winning probability at least 0.99969, and that there is a strategy for achieving the 256 tile guaranteed (with probability 1).

What problem does this paper attempt to address?

Playing 2048 With Reinforcement Learning

Shilun Li,Veronica Peng

DOI: https://doi.org/10.48550/arXiv.2110.10374

2021-10-20

Abstract:The game of 2048 is a highly addictive game. It is easy to learn the game, but hard to master as the created game revealed that only about 1% games out of hundreds million ever played have been won. In this paper, we would like to explore reinforcement learning techniques to win 2048. The approaches we have took include deep Q-learning and beam search, with beam search reaching 2048 28.5 of time.

Artificial Intelligence
On Reinforcement Learning for the Game of 2048

Hung Guei

DOI: https://doi.org/10.1109/TG.2021.3109887

2023-01-11

Abstract:2048 is a single-player stochastic puzzle game. This intriguing and addictive game has been popular worldwide and has attracted researchers to develop game-playing programs. Due to its simplicity and complexity, 2048 has become an interesting and challenging platform for evaluating the effectiveness of machine learning methods. This dissertation conducts comprehensive research on reinforcement learning and computer game algorithms for 2048. First, this dissertation proposes optimistic temporal difference learning, which significantly improves the quality of learning by employing optimistic initialization to encourage exploration for 2048. Furthermore, based on this approach, a state-of-the-art program for 2048 is developed, which achieves the highest performance among all learning-based programs, namely an average score of 625377 points and a rate of 72% for reaching 32768-tiles. Second, this dissertation investigates several techniques related to 2048, including the n-tuple network ensemble learning, Monte Carlo tree search, and deep reinforcement learning. These techniques are promising for further improving the performance of the current state-of-the-art program. Finally, this dissertation discusses pedagogical applications related to 2048 by proposing course designs and summarizing the teaching experience. The proposed course designs adopt 2048-like games as materials for beginners to learn reinforcement learning and computer game algorithms. The courses have been successfully applied to graduate-level students and received well by student feedback.

Machine Learning,Artificial Intelligence
On Covering Codes and Upper Bounds for the Dimension of Simple Games

Martin Olsen

DOI: https://doi.org/10.48550/arXiv.1609.04960

2016-09-16

Abstract:Consider a situation with $n$ agents or players where some of the players form a coalition with a certain collective objective. Simple games are used to model systems that can decide whether coalitions are successful (winning) or not (losing). A simple game can be viewed as a monotone boolean function. The dimension of a simple game is the smallest positive integer $d$ such that the simple game can be expressed as the intersection of $d$ threshold functions where each threshold function uses a threshold and $n$ weights. Taylor and Zwicker have shown that $d$ is bounded from above by the number of maximal losing coalitions. We present two new upper bounds both containing the Taylor/Zwicker-bound as a special case. The Taylor/Zwicker-bound imply an upper bound of ${n \choose n/2}$. We improve this upper bound significantly by showing constructively that $d$ is bounded from above by the cardinality of any binary covering code with length $n$ and covering radius $1$. This result supplements a recent result where Olsen et al. showed how to construct simple games with dimension $|C|$ for any binary constant weight SECDED code $C$ with length $n$. Our result represents a major step in the attempt to close the dimensionality gap for simple games.

Computer Science and Game Theory,Multiagent Systems
Ten times eighteen

Sebastian Böcker

DOI: https://doi.org/10.48550/arXiv.1209.1977

2012-09-10

Abstract:We consider the following simple game: We are given a table with ten slots indexed one to ten. In each of the ten rounds of the game, three dice are rolled and the numbers are added. We then put this number into any free slot. For each slot, we multiply the slot index with the number in this slot, and add up the products. The goal of the game is to maximize this score. In more detail, we play the game many times, and try to maximize the sum of scores or, equivalently, the expected score. We present a strategy to optimally play this game with respect to the expected score. We then modify our strategy so that we need only polynomial time and space. Finally, we show that knowing all ten rolls in advance, results in a relatively small increase in score. Although the game has a random component and requires a non-trivial strategy to be solved optimally, this strategy needs only polynomial time and space.

Discrete Mathematics,Data Structures and Algorithms
A Computational Tsirelson's Theorem for the Value of Compiled XOR Games

David Cui,Giulio Malavolta,Arthur Mehta,Anand Natarajan,Connor Paddock,Simon Schmidt,Michael Walter,Tina Zhang

2024-09-25

Abstract:Nonlocal games are a foundational tool for understanding entanglement and constructing quantum protocols in settings with multiple spatially separated quantum devices. In this work, we continue the study initiated by Kalai et al. (STOC '23) of compiled nonlocal games, played between a classical verifier and a single cryptographically limited quantum device. Our main result is that the compiler proposed by Kalai et al. is sound for any two-player XOR game. A celebrated theorem of Tsirelson shows that for XOR games, the quantum value is exactly given by a semidefinite program, and we obtain our result by showing that the SDP upper bound holds for the compiled game up to a negligible error arising from the compilation. This answers a question raised by Natarajan and Zhang (FOCS '23), who showed soundness for the specific case of the CHSH game. Using our techniques, we obtain several additional results, including (1) tight bounds on the compiled value of parallel-repeated XOR games, (2) operator self-testing statements for any compiled XOR game, and (3) a ``nice'' sum-of-squares certificate for any XOR game, from which operator rigidity is manifest.

Quantum Physics
Square++: Making a connection game win-lose complementary and playing-fair

Yi Yang,Shuigeng Zhou,Yuan Li

DOI: https://doi.org/10.1016/j.entcom.2012.10.004

IF: 2.072

2013-01-01

Entertainment Computing

Abstract:A connection game called Square++ is designed, which is played by almost similar rules to that of Hex but on a square board. Square++ belongs to the category of random-turn games, which is little explored. As the two players in Square++ act on different roles, we apply a biased coin to make the game playing fair. That is, one player has p chance to go, the other player has (1. -. p) chance to go.The challenging issue of this study is to find the exact value of p for a given size L of the board, such that both players have an equal chance to win by the best strategy. This p value is called fair probability p0.5(L). We first prove that the process of the game is equivalent to randomly filling the board by tossing the coin. We then design a dynamic programming algorithm to get high-accuracy values of p0.5(L) for L22, and estimate p0.5(L) for L>22 by numerical methods. Finally, we discuss the implementation issues of Square++, and introduce some of its variants. © 2012 International Federation for Information Processing.
Bounds for the Competition-Independence game on trees

Jan Petr,Julien Portier

2023-03-17

Abstract:In this paper we prove that Sweller has a strategy so that the Sweller-Start Competition-Independence game lasts at least $(5n+3)/13$ moves for every tree. Moreover, we show that there exist arbitrarily large trees such that the Sweller-Start Competition-Independence game lasts at most $(5n+26)/12$ moves, disproving a conjecture by Henning.

Combinatorics
A Constructive Winning Maker Strategy in the Maker-Breaker $C_4$-Game

Matthias Sowa,Anand Srivastav

2024-06-26

Abstract:Maker-Breaker subgraph games are among the most famous combinatorial games. For given $n,q \in \mathbb{N}$ and a subgraph $C$ of the complete graph $K_n$, the two players, called Maker and Breaker, alternately claim edges of $K_n$. In each round of the game Maker claims one edge and Breaker is allowed to claim up to $q$ edges. If Maker is able to claim all edges of a copy of $C$, he wins the game. Otherwise Breaker wins. In this work we introduce the first constructive strategy for Maker for the $C_4$-Maker-Breaker game and show that he can win the game if $q < 0.16 n^{2/3}$. According to the theorem of Bednarska and Luczak (2000) $n^{2/3}$ is asymptotically optimal for this game, but the constant given there for a random Maker strategy is magnitudes apart from our constant 0.16.

Combinatorics
Othello is Solved

Hiroki Takizawa

2024-01-03

Abstract:The game of Othello is one of the world's most complex and popular games that has yet to be computationally solved. Othello has roughly ten octodecillion (10 to the 58th power) possible game records and ten octillion (10 to the 28th power) possible game positions. The challenge of solving Othello, determining the outcome of a game with no mistake made by either player, has long been a grand challenge in computer science. This paper announces a significant milestone: Othello is now solved. It is computationally proved that perfect play by both players lead to a draw. Strong Othello software has long been built using heuristically designed search techniques. Solving a game provides a solution that enables the software to play the game perfectly.

Artificial Intelligence
Finding and Certifying (Near-)Optimal Strategies in Black-Box Extensive-Form Games

Brian Hu Zhang,Tuomas Sandholm

DOI: https://doi.org/10.48550/arXiv.2009.07384

2021-03-17

Abstract:Often -- for example in war games, strategy video games, and financial simulations -- the game is given to us only as a black-box simulator in which we can play it. In these settings, since the game may have unknown nature action distributions (from which we can only obtain samples) and/or be too large to expand fully, it can be difficult to compute strategies with guarantees on exploitability. Recent work \cite{Zhang20:Small} resulted in a notion of certificate for extensive-form games that allows exploitability guarantees while not expanding the full game tree. However, that work assumed that the black box could sample or expand arbitrary nodes of the game tree at any time, and that a series of exact game solves (via, for example, linear programming) can be conducted to compute the certificate. Each of those two assumptions severely restricts the practical applicability of that method. In this work, we relax both of the assumptions. We show that high-probability certificates can be obtained with a black box that can do nothing more than play through games, using only a regret minimizer as a subroutine. As a bonus, we obtain an equilibrium-finding algorithm with $\tilde O(1/\sqrt{T})$ convergence rate in the extensive-form game setting that does not rely on a sampling strategy with lower-bounded reach probabilities (which MCCFR assumes). We demonstrate experimentally that, in the black-box setting, our methods are able to provide nontrivial exploitability guarantees while expanding only a small fraction of the game tree.

Computer Science and Game Theory
Winning Strategies for Generalized Zeckendorf Game

Steven J. Miller,Eliel Sosis,Jingkai Ye

DOI: https://doi.org/10.48550/arXiv.2211.14973

2022-11-28

Abstract:Zeckendorf proved that every positive integer $n$ can be written uniquely as the sum of non-adjacent Fibonacci numbers; a similar result holds for other positive linear recurrence sequences. These legal decompositions can be used to construct a game that starts with a fixed integer $n$, and players take turns using moves relating to a given recurrence relation. The game eventually terminates in a unique legal decomposition, and the player who makes the final move wins. For the Fibonacci game, Player $2$ has the winning strategy for all $n>2$. We give a non-constructive proof that for the two-player $(c, k)$-nacci game, for all $k$ and sufficiently large $n$, Player $1$ has a winning strategy when $c$ is even and Player $2$ has a winning strategy when $c$ is odd. Interestingly, the player with the winning strategy can make a mistake as early as the $c + 1$ turn, in which case the other player gains the winning strategy. Furthermore, we proved that for the $(c, k)$-nacci game with players $p \ge c + 2$, no player has a winning strategy for any $n \ge 3c^2 + 6c + 3$. We find a stricter lower boundary, $n \ge 7$, in the case of the three-player $(1, 2)$-nacci game. Then we extend the result from the multiplayer game to multialliance games, showing which alliance has a winning strategy or when no winning strategy exists for some special cases of multialliance games.

Number Theory
The Cost of Derandomization: Computability or Competitiveness

XT Deng,S Mahajan

DOI: https://doi.org/10.1137/s0097539791202301

1997-01-01

SIAM Journal on Computing

Abstract:Recently, much work has been done in game theory towards understanding the bounded rationality of players in infinite games. This requires the strategies of realistic players to be restricted to have bounded resources of reasoning. (See [H. Simon, Decision and Organization, North--Holland, Amsterdam, 1972, pp. 161--176] for an extensive discussion; also see [X. Deng and C. H. Papadimitriou, Math. Oper. Res., 19 (1994), pp. 257--266], [C. Futia, J. Math. Econom., 4 (1977), pp. 289--299], [V. Knoblauch, Games Econom. Behav., 7 (1994), pp. 381--389], [E. Kalai and W. Stanford, Econometria, 56 (1988), pp. 397--410], [A. Neyman, Econom. Lett., 19 (1985), pp. 227--229], and [C. H. Papadimitriou, Game Theory Econom. Behav., 4 (1992), pp. 122--131].) In this paper, we discuss infinite two-person games, focusing on the case where our player follows a computable strategy and the adversary may use any strategy, which formulates the notion of computer against extremely formidable nature. In this context, we say that an infinite game is semicomputably determinate if either the adversary has a winning strategy or our player has a computable winning strategy.We show that, whereas all open games are semicomputably determinate, there is a semicomputably indeterminate closed game. Since we want to prove an indeterminacy result for closed games and since the adversary's strategy set is uncountable and our player's strategy set is countable, our proof for the indeterminacy result requires a new diagonalization technique, which might be useful in other similar cases. Our study of semicomputable games was inspired by online computing problems. In this direction, we discuss several possible applications to derandomization in online computing, with the restriction that the strategies of our player should be computable.We also study the power of randomization for the classical case where our player is allowed to play according to unrestricted strategies. An indeterminate game is obtained for which both players have a simple randomized winning strategy against all of the deterministic strategies of the opponent.
Quantum bounds for compiled XOR games and $d$-outcome CHSH games

Matilde Baroni,Quoc-Huy Vu,Boris Bourdoncle,Eleni Diamanti,Damian Markham,Ivan Šupić

2024-06-04

Abstract:Nonlocal games play a crucial role in quantum information theory and have numerous applications in certification and cryptographic protocols. Kalai et al. (STOC 2023) introduced a procedure to compile a nonlocal game into a single-prover interactive proof, using a quantum homomorphic encryption scheme, and showed that their compilation method preserves the classical bound of the game. Natarajan and Zhang (FOCS 2023) then showed that the quantum bound is preserved for the specific case of the CHSH game. Extending the proof techniques of Natarajan and Zhang, we show that the compilation procedure of Kalai et al. preserves the quantum bound for two classes of games: XOR games and d-outcome CHSH games. We also establish that, for any pair of qubit measurements, there exists an XOR game such that its optimal winning probability serves as a self-test for that particular pair of measurements.

Quantum Physics
At Most 43 Moves, At Least 29: Optimal Strategies and Bounds for Ultimate Tic-Tac-Toe

Guillaume Bertholon,Rémi Géraud-Stewart,Axel Kugelmann,Théo Lenoir,David Naccache

DOI: https://doi.org/10.48550/arXiv.2006.02353

2020-06-06

Abstract:Ultimate Tic-Tac-Toe is a variant of the well known tic-tac-toe (noughts and crosses) board game. Two players compete to win three aligned "fields", each of them being a tic-tac-toe game. Each move determines which field the next player must play in. We show that there exist a winning strategy for the first player, and therefore that there exist an optimal winning strategy taking at most 43 moves; that the second player can hold on at least 29 rounds; and identify any optimal strategy's first two moves.

Computer Science and Game Theory
Modeling Precomputation In Games Played Under Computational Constraints

Thomas Orton

DOI: https://doi.org/10.48550/arXiv.2105.09145

2021-05-19

Abstract:Understanding the properties of games played under computational constraints remains challenging. For example, how do we expect rational (but computationally bounded) players to play games with a prohibitively large number of states, such as chess? This paper presents a novel model for the precomputation (preparing moves in advance) aspect of computationally constrained games. A fundamental trade-off is shown between randomness of play, and susceptibility to precomputation, suggesting that randomization is necessary in games with computational constraints. We present efficient algorithms for computing how susceptible a strategy is to precomputation, and computing an $\epsilon$-Nash equilibrium of our model. Numerical experiments measuring the trade-off between randomness and precomputation are provided for Stockfish (a well-known chess playing algorithm).

Computer Science and Game Theory,Computational Complexity
Swim till You Sink: Computing the Limit of a Game

Rashida Hakim,Jason Milionis,Christos Papadimitriou,Georgios Piliouras

2024-08-21

Abstract:During 2023, two interesting results were proven about the limit behavior of game dynamics: First, it was shown that there is a game for which no dynamics converges to the Nash equilibria. Second, it was shown that the sink equilibria of a game adequately capture the limit behavior of natural game dynamics. These two results have created a need and opportunity to articulate a principled computational theory of the meaning of the game that is based on game dynamics. Given any game in normal form, and any prior distribution of play, we study the problem of computing the asymptotic behavior of a class of natural dynamics called the noisy replicator dynamics as a limit distribution over the sink equilibria of the game. When the prior distribution has pure strategy support, we prove this distribution can be computed efficiently, in near-linear time to the size of the best-response graph. When the distribution can be sampled -- for example, if it is the uniform distribution over all mixed strategy profiles -- we show through experiments that the limit distribution of reasonably large games can be estimated quite accurately through sampling and simulation.

Computer Science and Game Theory,Machine Learning,Theoretical Economics
Optimal Doubling Thresholds in Backgammon-like Stochastic Games

Haoru Ju,Daniel Leifer,Steven J. Miller,Sooraj A. Padmanabhan,Chenyang Sun,Luke Tichi,Benjamin Tocher,Kiley Wallace

2024-10-25

Abstract:We study variants of a stochastic game inspired by backgammon where players may propose to double the stake, with the game state dictated by a one-dimensional random walk. Our variants allow for different numbers of proposals and different multipliers to the stake. We determine the optimal game state for proposing and accepting, giving analytic solutions in many variants. We also introduce a 3-player generalization of the game and prove basic results about its behavior, in addition to providing a simulation.

Optimization and Control,Probability
A Note on Computational Complexity of Dou Shou Qi

Zhujun Zhang

DOI: https://doi.org/10.48550/arXiv.1904.13205

2019-04-27

Abstract:Dou Shou Qi is a Chinese strategy board game for two players. We use a EXPTIME-hardness framework to analyse computational complexity of the game. We construct all gadgets of the hardness framework. In conclusion, we prove that Dou Shou Qi is EXPTIME-complete.

Computational Complexity
The Maker-Breaker percolation game on a random board

Vojtěch Dvořák,Adva Mond,Victor Souza

2024-02-27

Abstract:The $(m,b)$ Maker-Breaker percolation game on $(\mathbb{Z}^2)_p$, introduced by Day and Falgas-Ravry, is played in the following way. Before the game starts, each edge of $\mathbb{Z}^2$ is removed independently with probability $1-p$. After that, Maker chooses a vertex $v_0$ to protect. Then, in each round Maker and Breaker claim respectively $m$ and $b$ unclaimed edges of $G$. Breaker wins if after the removal of the edges claimed by him the component of $v_0$ becomes finite, and Maker wins if she can indefinitely prevent Breaker from winning. We show that for any $p < 1$, Breaker almost surely has a wining strategy for the $(1,1)$ game on $(\mathbb{Z}^2)_p$. This fully answers a question of Day and Falgas-Ravry, who showed that for $p = 1$ Maker has a winning strategy for the $(1,1)$ game. Further, we show that in the $(2,1)$ game on $(\mathbb{Z}^2)_p$ Maker almost surely has a winning strategy whenever $p > 0.9402$, while Breaker almost surely has a winning strategy whenever $p < 0.5278$. This shows that the threshold value of $p$ above which Maker has a winning strategy for the $(2,1)$ game on $\mathbb{Z}^2$ is non-trivial. In fact, we prove similar results in various settings, including other lattices and biases $(m,b)$. These results extend also to the most general case, which we introduce, where each edge is given to Maker with probability $\alpha$ and to Breaker with probability $\beta$ before the game starts.

Combinatorics,Probability
Computability of simple games: A complete investigation of the sixty-four possibilities

Masahiro Kumabe,H. Reiju Mihara

DOI: https://doi.org/10.1016/j.jmateco.2010.12.003

2011-02-20

Abstract:Classify simple games into sixteen "types" in terms of the four conventional axioms: monotonicity, properness, strongness, and nonweakness. Further classify them into sixty-four classes in terms of finiteness (existence of a finite carrier) and algorithmic computability. For each such class, we either show that it is empty or give an example of a game belonging to it. We observe that if a type contains an infinite game, then it contains both computable ones and noncomputable ones. This strongly suggests that computability is logically, as well as conceptually, unrelated to the conventional axioms.

Computer Science and Game Theory,Logic in Computer Science,Logic

Computational bounds for the 2048 game

Playing 2048 With Reinforcement Learning

On Reinforcement Learning for the Game of 2048

On Covering Codes and Upper Bounds for the Dimension of Simple Games

Ten times eighteen

A Computational Tsirelson's Theorem for the Value of Compiled XOR Games

Square++: Making a connection game win-lose complementary and playing-fair

Bounds for the Competition-Independence game on trees

A Constructive Winning Maker Strategy in the Maker-Breaker $C_4$-Game

Othello is Solved

Finding and Certifying (Near-)Optimal Strategies in Black-Box Extensive-Form Games

Winning Strategies for Generalized Zeckendorf Game

The Cost of Derandomization: Computability or Competitiveness

Quantum bounds for compiled XOR games and $d$-outcome CHSH games

At Most 43 Moves, At Least 29: Optimal Strategies and Bounds for Ultimate Tic-Tac-Toe

Modeling Precomputation In Games Played Under Computational Constraints

Swim till You Sink: Computing the Limit of a Game

Optimal Doubling Thresholds in Backgammon-like Stochastic Games

A Note on Computational Complexity of Dou Shou Qi

The Maker-Breaker percolation game on a random board

Computability of simple games: A complete investigation of the sixty-four possibilities