Reimagining chess with AlphaZero

Nenad Tomašev,Ulrich Paquet,Demis Hassabis,Vladimir Kramnik

DOI: https://doi.org/10.1145/3460349

IF: 22.7

2022-02-01

Communications of the ACM

Abstract:AI is driving the next evolution of chess, giving players a glimpse into the game's future.

computer science, theory & methods, software engineering, hardware & architecture

What problem does this paper attempt to address?

Acquisition of chess knowledge in AlphaZero

Thomas McGrath,Andrei Kapishnikov,Nenad Tomašev,Adam Pearce,Martin Wattenberg,Demis Hassabis,Been Kim,Ulrich Paquet,Vladimir Kramnik

DOI: https://doi.org/10.1073/pnas.2206625119

2022-11-15

Abstract:We analyze the knowledge acquired by AlphaZero, a neural network engine that learns chess solely by playing against itself yet becomes capable of outperforming human chess players. Although the system trains without access to human games or guidance, it appears to learn concepts analogous to those used by human chess players. We provide two lines of evidence. Linear probes applied to AlphaZero's internal state enable us to quantify when and where such concepts are represented in the network. We also describe a behavioral analysis of opening play, including qualitative commentary by a former world chess champion.

multidisciplinary sciences
One giant step for a chess-playing machine

Steven Strogatz

2018-12-26

Abstract:Ayear earlier, on Dec. 5, 2017, the team had stunned the chess world with its announcement of AlphaZero, a machine-learning algorithm that had mastered not only chess but shogi, or Japanese chess, and Go. The algorithm started with no knowledge of the games beyond their basic rules. It then played againstitself millions of times and learned from its mistakes. In a matter of hours, the algorithm became the best player, human or computer, the world has ever seen.The details of AlphaZero's achievements and inner workings have now been formally peer-reviewed and published in the journal Science this month. The new paper addresses several serious criticisms of the original claim.(Among other things, it was hard to tell whether AlphaZero was playing its chosen opponent, a computational beast named Stockfish, with total fairness.) Consider those concerns dispelled. AlphaZero has not grown stronger in the …
Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess

Nenad Tomašev,Ulrich Paquet,Demis Hassabis,Vladimir Kramnik

DOI: https://doi.org/10.48550/arXiv.2009.04374

IF: 14.4

2020-09-09

Artificial Intelligence

Abstract:It is non-trivial to design engaging and balanced sets of game rules. Modern chess has evolved over centuries, but without a similar recourse to history, the consequences of rule changes to game dynamics are difficult to predict. AlphaZero provides an alternative in silico means of game balance assessment. It is a system that can learn near-optimal strategies for any rule set from scratch, without any human supervision, by continually learning from its own experience. In this study we use AlphaZero to creatively explore and design new chess variants. There is growing interest in chess variants like Fischer Random Chess, because of classical chess's voluminous opening theory, the high percentage of draws in professional play, and the non-negligible number of games that end while both players are still in their home preparation. We compare nine other variants that involve atomic changes to the rules of chess. The changes allow for novel strategic and tactical patterns to emerge, while keeping the games close to the original. By learning near-optimal strategies for each variant with AlphaZero, we determine what games between strong human players might look like if these variants were adopted. Qualitatively, several variants are very dynamic. An analytic comparison show that pieces are valued differently between variants, and that some variants are more decisive than classical chess. Our findings demonstrate the rich possibilities that lie beyond the rules of modern chess.
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

David Silver,Thomas Hubert,Julian Schrittwieser,Ioannis Antonoglou,Matthew Lai,Arthur Guez,Marc Lanctot,Laurent Sifre,Dharshan Kumaran,Thore Graepel,Timothy Lillicrap,Karen Simonyan,Demis Hassabis

DOI: https://doi.org/10.1126/science.aar6404

IF: 56.9

2018-12-07

Science

Abstract:One program to rule them all Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118

multidisciplinary sciences
Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero

Lisa Schut,Nenad Tomasev,Tom McGrath,Demis Hassabis,Ulrich Paquet,Been Kim

2023-10-25

Abstract:Artificial Intelligence (AI) systems have made remarkable progress, attaining super-human performance across various domains. This presents us with an opportunity to further human knowledge and improve human expert performance by leveraging the hidden knowledge encoded within these highly performant AI systems. Yet, this knowledge is often hard to extract, and may be hard to understand or learn from. Here, we show that this is possible by proposing a new method that allows us to extract new chess concepts in AlphaZero, an AI system that mastered the game of chess via self-play without human supervision. Our analysis indicates that AlphaZero may encode knowledge that extends beyond the existing human knowledge, but knowledge that is ultimately not beyond human grasp, and can be successfully learned from. In a human study, we show that these concepts are learnable by top human experts, as four top chess grandmasters show improvements in solving the presented concept prototype positions. This marks an important first milestone in advancing the frontier of human knowledge by leveraging AI; a development that could bear profound implications and help us shape how we interact with AI systems across many AI applications.

Human-Computer Interaction,Machine Learning
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

David Silver,Thomas Hubert,Julian Schrittwieser,Ioannis Antonoglou,Matthew Lai,Arthur Guez,Marc Lanctot,Laurent Sifre,Dharshan Kumaran,Thore Graepel,Timothy Lillicrap,Karen Simonyan,Demis Hassabis

DOI: https://doi.org/10.48550/arXiv.1712.01815

IF: 14.4

2017-12-05

Artificial Intelligence

Abstract:The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.
Diversifying AI: Towards Creative Chess with AlphaZero

Tom Zahavy,Vivek Veeriah,Shaobo Hou,Kevin Waugh,Matthew Lai,Edouard Leurent,Nenad Tomasev,Lisa Schut,Demis Hassabis,Satinder Singh

2024-07-31

Abstract:In recent years, Artificial Intelligence (AI) systems have surpassed human intelligence in a variety of computational tasks. However, AI systems, like humans, make mistakes, have blind spots, hallucinate, and struggle to generalize to new situations. This work explores whether AI can benefit from creative decision-making mechanisms when pushed to the limits of its computational rationality. In particular, we investigate whether a team of diverse AI systems can outperform a single AI in challenging tasks by generating more ideas as a group and then selecting the best ones. We study this question in the game of chess, the so-called drosophila of AI. We build on AlphaZero (AZ) and extend it to represent a league of agents via a latent-conditioned architecture, which we call AZ_db. We train AZ_db to generate a wider range of ideas using behavioral diversity techniques and select the most promising ones with sub-additive planning. Our experiments suggest that AZ_db plays chess in diverse ways, solves more puzzles as a group and outperforms a more homogeneous team. Notably, AZ_db solves twice as many challenging puzzles as AZ, including the challenging Penrose positions. When playing chess from different openings, we notice that players in AZ_db specialize in different openings, and that selecting a player for each opening using sub-additive planning results in a 50 Elo improvement over AZ. Our findings suggest that diversity bonuses emerge in teams of AI agents, just as they do in teams of humans and that diversity is a valuable asset in solving computationally hard problems.

Artificial Intelligence,Machine Learning
Neural Networks for Chess

Dominik Klein

DOI: https://doi.org/10.48550/arXiv.2209.01506

2022-09-04

Abstract:AlphaZero, Leela Chess Zero and Stockfish NNUE revolutionized Computer Chess. This book gives a complete introduction into the technical inner workings of such engines. The book is split into four main chapters -- excluding chapter 1 (introduction) and chapter 6 (conclusion): Chapter 2 introduces neural networks and covers all the basic building blocks that are used to build deep networks such as those used by AlphaZero. Contents include the perceptron, back-propagation and gradient descent, classification, regression, multilayer perceptron, vectorization techniques, convolutional networks, squeeze and excitation networks, fully connected networks, batch normalization and rectified linear units, residual layers, overfitting and underfitting. Chapter 3 introduces classical search techniques used for chess engines as well as those used by AlphaZero. Contents include minimax, alpha-beta search, and Monte Carlo tree search. Chapter 4 shows how modern chess engines are designed. Aside from the ground-breaking AlphaGo, AlphaGo Zero and AlphaZero we cover Leela Chess Zero, Fat Fritz, Fat Fritz 2 and Efficiently Updatable Neural Networks (NNUE) as well as Maia. Chapter 5 is about implementing a miniaturized AlphaZero. Hexapawn, a minimalistic version of chess, is used as an example for that. Hexapawn is solved by minimax search and training positions for supervised learning are generated. Then as a comparison, an AlphaZero-like training loop is implemented where training is done via self-play combined with reinforcement learning. Finally, AlphaZero-like training and supervised training are compared.

Machine Learning,Artificial Intelligence
Computers and chess masters: The role of AI in transforming elite human performance

Merim Bilalić,Mario Graf,Nemanja Vaci

DOI: https://doi.org/10.1111/bjop.12750

IF: 4.981

2024-12-07

British Journal of Psychology

Abstract:Advances in Artificial Intelligence (AI) have made significant strides in recent years, often supplementing rather than replacing human performance. The extent of their assistance at the highest levels of human performance remains unclear. We analyse over 11.6 million decisions of elite chess players, a domain commonly used as a testbed for AI and psychology due to its complexity and objective assessment. We investigated the impact of two AI chess revolutions: the first in the late 1990s with the rise of powerful PCs and internet access and the second in the late 2010s with deep learning‐powered chess engines. The rate of human improvement mirrored AI advancements, but contrary to expectations, the quality of decisions mostly improved steadily over four decades, irrespective of age, with no distinct periods of rapid improvement. Only the youngest top players saw marked gains in the late 1990s, likely due to better access to knowledge and computers. Surprisingly, the recent wave of neural network‐powered engines has not significantly impacted the best players – at least, not yet. Our research highlights AI's potential to enhance human capability in complex tasks, given the right conditions, even among the most elite performers.

psychology, multidisciplinary
Chess AI: Competing Paradigms for Machine Intelligence

Shiva Maharaj,Nick Polson,Alex Turk

DOI: https://doi.org/10.3390/e24040550

IF: 2.738

2022-04-14

Entropy

Abstract:Endgame studies have long served as a tool for testing human creativity and intelligence. We find that they can serve as a tool for testing machine ability as well. Two of the leading chess engines, Stockfish and Leela Chess Zero (LCZero), employ significantly different methods during play. We use Plaskett’s Puzzle, a famous endgame study from the late 1970s, to compare the two engines. Our experiments show that Stockfish outperforms LCZero on the puzzle. We examine the algorithmic differences between the engines and use our observations as a basis for carefully interpreting the test results. Drawing inspiration from how humans solve chess problems, we ask whether machines can possess a form of imagination. On the theoretical side, we describe how Bellman’s equation may be applied to optimize the probability of winning. To conclude, we discuss the implications of our work on artificial intelligence (AI) and artificial general intelligence (AGI), suggesting possible avenues for future research.

physics, multidisciplinary
AlphaZero Gomoku

Wen Liang,Chao Yu,Brian Whiteaker,Inyoung Huh,Hua Shao,Youzhi Liang

DOI: https://doi.org/10.48550/arXiv.2309.01294

IF: 14.4

2023-09-04

Artificial Intelligence

Abstract:In the past few years, AlphaZero's exceptional capability in mastering intricate board games has garnered considerable interest. Initially designed for the game of Go, this revolutionary algorithm merges deep learning techniques with the Monte Carlo tree search (MCTS) to surpass earlier top-tier methods. In our study, we broaden the use of AlphaZero to Gomoku, an age-old tactical board game also referred to as "Five in a Row." Intriguingly, Gomoku has innate challenges due to a bias towards the initial player, who has a theoretical advantage. To add value, we strive for a balanced game-play. Our tests demonstrate AlphaZero's versatility in adapting to games other than Go. MCTS has become a predominant algorithm for decision processes in intricate scenarios, especially board games. MCTS creates a search tree by examining potential future actions and uses random sampling to predict possible results. By leveraging the best of both worlds, the AlphaZero technique fuses deep learning from Reinforcement Learning with the balancing act of MCTS, establishing a fresh standard in game-playing AI. Its triumph is notably evident in board games such as Go, chess, and shogi.
Mastering NIM and Impartial Games with Weak Neural Networks: An AlphaZero-inspired Multi-Frame Approach

Søren Riis

2024-11-10

Abstract:This paper provides a theoretical framework that validates and explains the results in the work with Bei Zhou experimentally finding that AlphaZero-style reinforcement learning algorithms struggle to learn optimal play in NIM, a canonical impartial game proposed as an AI challenge by Harvey Friedman in 2017. Our analysis resolves a controversy around these experimental results, which revealed unexpected difficulties in learning NIM despite its mathematical simplicity compared to games like chess and Go. Our key contributions are as follows: We prove that by incorporating recent game history, these limited AlphaZero models can, in principle, achieve optimal play in NIM. We introduce a novel search strategy where roll-outs preserve game-theoretic values during move selection, guided by a specialised policy network. We provide constructive proofs showing that our approach enables optimal play within the $\text{AC}^0$ complexity class despite the theoretical limitations of these networks. This research demonstrates how constrained neural networks when properly designed, can achieve sophisticated decision-making even in domains where their basic computational capabilities appear insufficient.

Artificial Intelligence
DeepMind AI is a games master

Matthew Sparkes

DOI: https://doi.org/10.1016/s0262-4079(23)02185-1

2023-11-01

Abstract:System that beats humans at chess and poker could be a step towards general intelligence

multidisciplinary sciences
Mastering Atari, Go, chess and shogi by planning with a learned model

Julian Schrittwieser,Ioannis Antonoglou,Thomas Hubert,Karen Simonyan,Laurent Sifre,Simon Schmitt,Arthur Guez,Edward Lockhart,Demis Hassabis,Thore Graepel,Timothy Lillicrap,David Silver

DOI: https://doi.org/10.1038/s41586-020-03051-4

IF: 64.8

2020-12-23

Nature

Abstract:Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess<a href="/articles/s41586-020-03051-4#ref-CR1">1</a> and Go<a href="/articles/s41586-020-03051-4#ref-CR2">2</a>, where a perfect simulator is available. However, in real-world problems, the dynamics governing the environment are often complex and unknown. Here we present the MuZero algorithm, which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. The MuZero algorithm learns an iterable model that produces predictions relevant to planning: the action-selection policy, the value function and the reward. When evaluated on 57 different Atari games<a href="/articles/s41586-020-03051-4#ref-CR3">3</a>—the canonical video game environment for testing artificial intelligence techniques, in which model-based planning approaches have historically struggled<a href="/articles/s41586-020-03051-4#ref-CR4">4</a>—the MuZero algorithm achieved state-of-the-art performance. When evaluated on Go, chess and shogi—canonical environments for high-performance planning—the MuZero algorithm matched, without any knowledge of the game dynamics, the superhuman performance of the AlphaZero algorithm<a href="/articles/s41586-020-03051-4#ref-CR5">5</a> that was supplied with the rules of the game.

multidisciplinary sciences
Application And Challenges of Alphazero Algorithm in Chinese Chess

Xiaxi Li

DOI: https://doi.org/10.54097/e6373366

2024-01-26

Abstract:In the field of artificial intelligence, AlphaZero has achieved significant success in games like Go and Shogi. However, its application in Chinese Chess remains relatively unexplored. This study aims to unveil the potential and limitations of AlphaZero in Chinese Chess and explore strategies for optimizing its performance. In this paper, we begin by reviewing the fundamental principles and algorithmic structure of AlphaZero, emphasizing its deep reinforcement learning and self-play characteristics. Subsequently, we delve into the specific application of AlphaZero in Chinese Chess, encompassing aspects such as chess rule representation, training procedures, position evaluation, and strategy prediction. By analyzing successful instances of AlphaZero in Chinese Chess, we showcase its potential in the domain of board games. Furthermore, this paper elucidates the challenges that AlphaZero encounters in the context of Chinese Chess, including issues related to computational complexity and position representation. We particularly underscore the potential cost escalation due to high computational resource requirements and the risk of data loss or excessive time consumption associated with extended computation periods. IN conclusion, this research provides valuable insights into the application of AlphaZero in the realm of Chinese Chess and offers recommendations for future research directions to further expand its utility in this domain.
Schlau, schlauer, am schlausten: AlphaGo Zero

Janosch Deeg

DOI: https://doi.org/10.1007/978-3-662-62492-0_11

2022-01-01

Abstract:Frühjahr 2016: Die künstliche Intelligenz (KI) AlphaGo war bereit für das lange erwartete Duell; im Strategiespiel Go trat sie mehrmals gegen den weltbesten Spieler Lee Sedol aus Korea an. Das Ergebnis fiel mehr als deutlich aus: Der schlaue Algorithmus, der ähnlich wie ein neuronales Netzwerk funktioniert, gewann vier von fünf Duellen. KI schlägt Mensch (hier ein Bericht nach der ersten Partie).
Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers

Johannes Czech,Jannis Blüml,Kristian Kersting,Hedinn Steingrimsson

2024-08-21

Abstract:While transformers have gained recognition as a versatile tool for artificial intelligence (AI), an unexplored challenge arises in the context of chess - a classical AI benchmark. Here, incorporating Vision Transformers (ViTs) into AlphaZero is insufficient for chess mastery, mainly due to ViTs' computational limitations. The attempt to optimize their efficiency by combining MobileNet and NextViT outperformed AlphaZero by about 30 Elo. However, we propose a practical improvement that involves a simple change in the input representation and value loss functions. As a result, we achieve a significant performance boost of up to 180 Elo points beyond what is currently achievable with AlphaZero in chess. In addition to these improvements, our experimental results using the Integrated Gradient technique confirm the effectiveness of the newly introduced features.

Artificial Intelligence
Mastering the game of Go without human knowledge

David Silver,Julian Schrittwieser,Karen Simonyan,Ioannis Antonoglou,Aja Huang,Arthur Guez,Thomas Hubert,Lucas Baker,Matthew Lai,Adrian Bolton,Yutian Chen,Timothy Lillicrap,Fan Hui,Laurent Sifre,George van den Driessche,Thore Graepel,Demis Hassabis

DOI: https://doi.org/10.1038/nature24270

IF: 64.8

2017-10-01

Nature

Abstract:A long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains. Recently, AlphaGo became the first program to defeat a world champion in the game of Go. The tree search in AlphaGo evaluated positions and selected moves using deep neural networks. These neural networks were trained by supervised learning from human expert moves, and by reinforcement learning from self-play. Here we introduce an algorithm based solely on reinforcement learning, without human data, guidance or domain knowledge beyond game rules. AlphaGo becomes its own teacher: a neural network is trained to predict AlphaGo’s own move selections and also the winner of AlphaGo’s games. This neural network improves the strength of the tree search, resulting in higher quality move selection and stronger self-play in the next iteration. Starting tabula rasa, our new program AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo.

multidisciplinary sciences
Enhancing Chess Reinforcement Learning with Graph Representation

Tomas Rigaux,Hisashi Kashima

2024-10-31

Abstract:Mastering games is a hard task, as games can be extremely complex, and still fundamentally different in structure from one another. While the AlphaZero algorithm has demonstrated an impressive ability to learn the rules and strategy of a large variety of games, ranging from Go and Chess, to Atari games, its reliance on extensive computational resources and rigid Convolutional Neural Network (CNN) architecture limits its adaptability and scalability. A model trained to play on a $19\times 19$ Go board cannot be used to play on a smaller $13\times 13$ board, despite the similarity between the two Go variants. In this paper, we focus on Chess, and explore using a more generic Graph-based Representation of a game state, rather than a grid-based one, to introduce a more general architecture based on Graph Neural Networks (GNN). We also expand the classical Graph Attention Network (GAT) layer to incorporate edge-features, to naturally provide a generic policy output format. Our experiments, performed on smaller networks than the initial AlphaZero paper, show that this new architecture outperforms previous architectures with a similar number of parameters, being able to increase playing strength an order of magnitude faster. We also show that the model, when trained on a smaller $5\times 5$ variant of chess, is able to be quickly fine-tuned to play on regular $8\times 8$ chess, suggesting that this approach yields promising generalization abilities. Our code is available at <a class="link-external link-https" href="https://github.com/akulen/AlphaGateau" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence

Reimagining chess with AlphaZero

Acquisition of chess knowledge in AlphaZero

One giant step for a chess-playing machine

Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

Diversifying AI: Towards Creative Chess with AlphaZero

Neural Networks for Chess

Computers and chess masters: The role of AI in transforming elite human performance

Chess AI: Competing Paradigms for Machine Intelligence

AlphaZero Gomoku

Mastering NIM and Impartial Games with Weak Neural Networks: An AlphaZero-inspired Multi-Frame Approach

DeepMind AI is a games master

Mastering Atari, Go, chess and shogi by planning with a learned model

Application And Challenges of Alphazero Algorithm in Chinese Chess

Schlau, schlauer, am schlausten: AlphaGo Zero

Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to Transformers

Mastering the game of Go without human knowledge

Enhancing Chess Reinforcement Learning with Graph Representation