Google AI algorithm masters ancient game of Go

Elizabeth Gibney

DOI: https://doi.org/10.1038/529445a

IF: 64.8

2016-01-27

Nature

Abstract:Deep-learning software defeats human professional for first time.

multidisciplinary sciences

What problem does this paper attempt to address?

Google reveals secret test of AI bot to beat top Go players

Elizabeth Gibney

DOI: https://doi.org/10.1038/nature.2017.21253

IF: 64.8

2017-01-01

Nature

Abstract:Updated version of DeepMind's AlphaGo program behind mystery online competitor.

multidisciplinary sciences
Mastering the game of Go without human knowledge

David Silver,Julian Schrittwieser,Karen Simonyan,Ioannis Antonoglou,Aja Huang,Arthur Guez,Thomas Hubert,Lucas Baker,Matthew Lai,Adrian Bolton,Yutian Chen,Timothy Lillicrap,Fan Hui,Laurent Sifre,George van den Driessche,Thore Graepel,Demis Hassabis

DOI: https://doi.org/10.1038/nature24270

IF: 64.8

2017-10-01

Nature

Abstract:A long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains. Recently, AlphaGo became the first program to defeat a world champion in the game of Go. The tree search in AlphaGo evaluated positions and selected moves using deep neural networks. These neural networks were trained by supervised learning from human expert moves, and by reinforcement learning from self-play. Here we introduce an algorithm based solely on reinforcement learning, without human data, guidance or domain knowledge beyond game rules. AlphaGo becomes its own teacher: a neural network is trained to predict AlphaGo’s own move selections and also the winner of AlphaGo’s games. This neural network improves the strength of the tree search, resulting in higher quality move selection and stronger self-play in the next iteration. Starting tabula rasa, our new program AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo.

multidisciplinary sciences
Mastering the game of Go with deep neural networks and tree search

David Silver,Aja Huang,Chris J. Maddison,Arthur Guez,Laurent Sifre,George van den Driessche,Julian Schrittwieser,Ioannis Antonoglou,Veda Panneershelvam,Marc Lanctot,Sander Dieleman,Dominik Grewe,John Nham,Nal Kalchbrenner,Ilya Sutskever,Timothy Lillicrap,Madeleine Leach,Koray Kavukcuoglu,Thore Graepel,Demis Hassabis

DOI: https://doi.org/10.1038/nature16961

IF: 64.8

2016-01-27

Nature

Abstract:The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

multidisciplinary sciences
When Doctors Meet with AlphaGo: Potential Application of Machine Learning to Clinical Medicine.

Zhongheng Zhang

DOI: https://doi.org/10.21037/atm.2016.03.25

IF: 3.616

2016-01-01

Annals of Translational Medicine

Abstract:March is a reluctant beginning of spring. After suddenly getting warmer, the weather has turned cold again. Yesterday, the friend circle of Wechat was full of a breaking news that the Google’s artificial intelligence AlphaGo beat human Go champion Lee Sedol. Or at least it was a temporary lead because there will be several rounds in following days. The news was a headline because it was related to the debate on whether artificial intelligence could defeat human brain. Many cultural celebrities posted their comments on the news. Shen Lei, a writer who is famous for his interpretation on Jin Yong’s Kung Fu novels, stated that: “ The last warrior falls. ” Tang Fenng also posted on his Wechat that: “ All calculations and computations can be left for machines from now on. What we humans can do are: burning incense, sipping tea, Washing inkstone, playing the Chinese zither…. ” In other words, all works with rules can be completed by machines, and humans have to retreat to their emotional domain. Why is it the Go, instead of other chesses such as Modern Ludo and animal checker, or other intellectual activities? That is because the Go represent the most complicated intellectual activity. The chessboard of Go is made up of 361 cross points, which in turn can generate numerous chess compositions. There are around 10 170 possible board configurations in the average 150-move game (1)! After all, this is much challenging than medical decision making. Two decades ago, the IBM supercomputer Deep Blue defeated the world chess champion Garry Kasparov (2). However, the chess is slightly inferior than Go in its complexity. Thus the AlphaGo’s victory has more far-reaching significance.
DeepMind AI is a games master

Matthew Sparkes

DOI: https://doi.org/10.1016/s0262-4079(23)02185-1

2023-11-01

Abstract:System that beats humans at chess and poker could be a step towards general intelligence

multidisciplinary sciences
Human vs. Computer Go: Review and Prospect

Chang-Shing Lee,Mei-Hui Wang,Shi-Jim Yen,Ting-Han Wei,I-Chen Wu,Ping-Chiang Chou,Chun-Hsun Chou,Ming-Wan Wang,Tai-Hsiung Yang

DOI: https://doi.org/10.1109/MCI.2016.2572559

2016-06-07

Abstract:The Google DeepMind challenge match in March 2016 was a historic achievement for computer Go development. This article discusses the development of computational intelligence (CI) and its relative strength in comparison with human intelligence for the game of Go. We first summarize the milestones achieved for computer Go from 1998 to 2016. Then, the computer Go programs that have participated in previous IEEE CIS competitions as well as methods and techniques used in AlphaGo are briefly introduced. Commentaries from three high-level professional Go players on the five AlphaGo versus Lee Sedol games are also included. We conclude that AlphaGo beating Lee Sedol is a huge achievement in artificial intelligence (AI) based largely on CI methods. In the future, powerful computer Go programs such as AlphaGo are expected to be instrumental in promoting Go education and AI real-world applications.

Artificial Intelligence
[Deep Learning and AlphaGo]

Hiroyuki Yoshida

DOI: https://doi.org/10.11477/mf.1416201340

Abstract:When the Information Processing Society of Japan (IPSJ) declared the end of the challenge to the Japanese Shogi Association in 2015, the belief was that it would take more than 20 years for computers to catch up with human professional Go players, as Go is far more complex than shogi from a combinatorial perspective. However, AlphaGo, whose paper was just published the same year, beat the Go world champion in all three games in 2017. In this paper, we describe the artificial intelligence methods adopted by AlphaGo, especially deep learning, and consider their relationship with neuroscience.
DeepMind AI creates algorithms that sort data faster than those built by people

Matthew Hutson

DOI: https://doi.org/10.1038/d41586-023-01883-4

IF: 64.8

2023-06-08

Nature

Abstract:The technology developed by DeepMind that plays Go and chess can also help to write code. The technology developed by DeepMind that plays Go and chess can also help to write code.

multidisciplinary sciences
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

David Silver,Thomas Hubert,Julian Schrittwieser,Ioannis Antonoglou,Matthew Lai,Arthur Guez,Marc Lanctot,Laurent Sifre,Dharshan Kumaran,Thore Graepel,Timothy Lillicrap,Karen Simonyan,Demis Hassabis

DOI: https://doi.org/10.48550/arXiv.1712.01815

IF: 14.4

2017-12-05

Artificial Intelligence

Abstract:The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.
AI cracks hard maths

Alex Wilkins

DOI: https://doi.org/10.1016/s0262-4079(24)00157-x

IF: 0.906

2024-01-28

The New Scientist

Abstract:An AI from Google DeepMind scores almost as well as the best students on geometry questions from the International Mathematical Olympiad, finds Alex Wilkins

multidisciplinary sciences
Teaching Deep Convolutional Neural Networks to Play Go

Christopher Clark,Amos Storkey

DOI: https://doi.org/10.48550/arXiv.1412.3409

2015-01-27

Abstract:Mastering the game of Go has remained a long standing challenge to the field of AI. Modern computer Go systems rely on processing millions of possible future positions to play well, but intuitively a stronger and more 'humanlike' way to play the game would be to rely on pattern recognition abilities rather then brute force computation. Following this sentiment, we train deep convolutional neural networks to play Go by training them to predict the moves made by expert Go players. To solve this problem we introduce a number of novel techniques, including a method of tying weights in the network to 'hard code' symmetries that are expect to exist in the target function, and demonstrate in an ablation study they considerably improve performance. Our final networks are able to achieve move prediction accuracies of 41.1% and 44.4% on two different Go datasets, surpassing previous state of the art on this task by significant margins. Additionally, while previous move prediction programs have not yielded strong Go playing programs, we show that the networks trained in this work acquired high levels of skill. Our convolutional neural networks can consistently defeat the well known Go program GNU Go, indicating it is state of the art among programs that do not use Monte Carlo Tree Search. It is also able to win some games against state of the art Go playing program Fuego while using a fraction of the play time. This success at playing Go indicates high level principles of the game were learned.

Artificial Intelligence,Machine Learning,Neural and Evolutionary Computing
Very simple statistical evidence that AlphaGo has exceeded human limits in playing GO game

Okyu Kwon

DOI: https://doi.org/10.48550/arXiv.2002.11107

2020-02-25

Abstract:Deep learning technology is making great progress in solving the challenging problems of artificial intelligence, hence machine learning based on artificial neural networks is in the spotlight again. In some areas, artificial intelligence based on deep learning is beyond human capabilities. It seemed extremely difficult for a machine to beat a human in a Go game, but AlphaGo has shown to beat a professional player in the game. By looking at the statistical distribution of the distance in which the Go stones are laid in succession, we find a clear trace that Alphago has surpassed human abilities. The AlphaGo than professional players and professional players than ordinary players shows the laying of stones in the distance becomes more frequent. In addition, AlphaGo shows a much more pronounced difference than that of ordinary players and professional players.

Artificial Intelligence
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

David Silver,Thomas Hubert,Julian Schrittwieser,Ioannis Antonoglou,Matthew Lai,Arthur Guez,Marc Lanctot,Laurent Sifre,Dharshan Kumaran,Thore Graepel,Timothy Lillicrap,Karen Simonyan,Demis Hassabis

DOI: https://doi.org/10.1126/science.aar6404

IF: 56.9

2018-12-07

Science

Abstract:One program to rule them all Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118

multidisciplinary sciences
AlphaZero Gomoku

Wen Liang,Chao Yu,Brian Whiteaker,Inyoung Huh,Hua Shao,Youzhi Liang

DOI: https://doi.org/10.48550/arXiv.2309.01294

IF: 14.4

2023-09-04

Artificial Intelligence

Abstract:In the past few years, AlphaZero's exceptional capability in mastering intricate board games has garnered considerable interest. Initially designed for the game of Go, this revolutionary algorithm merges deep learning techniques with the Monte Carlo tree search (MCTS) to surpass earlier top-tier methods. In our study, we broaden the use of AlphaZero to Gomoku, an age-old tactical board game also referred to as "Five in a Row." Intriguingly, Gomoku has innate challenges due to a bias towards the initial player, who has a theoretical advantage. To add value, we strive for a balanced game-play. Our tests demonstrate AlphaZero's versatility in adapting to games other than Go. MCTS has become a predominant algorithm for decision processes in intricate scenarios, especially board games. MCTS creates a search tree by examining potential future actions and uses random sampling to predict possible results. By leveraging the best of both worlds, the AlphaZero technique fuses deep learning from Reinforcement Learning with the balancing act of MCTS, establishing a fresh standard in game-playing AI. Its triumph is notably evident in board games such as Go, chess, and shogi.
Schlau, schlauer, am schlausten: AlphaGo Zero

Janosch Deeg

DOI: https://doi.org/10.1007/978-3-662-62492-0_11

2022-01-01

Abstract:Frühjahr 2016: Die künstliche Intelligenz (KI) AlphaGo war bereit für das lange erwartete Duell; im Strategiespiel Go trat sie mehrmals gegen den weltbesten Spieler Lee Sedol aus Korea an. Das Ergebnis fiel mehr als deutlich aus: Der schlaue Algorithmus, der ähnlich wie ein neuronales Netzwerk funktioniert, gewann vier von fünf Duellen. KI schlägt Mensch (hier ein Bericht nach der ersten Partie).
“Naughty AlphaGo”: Transforming the Game of Computer Go into an Emotional Tangible Playground

Shuchang Xu,Yilin Zhu,Chunhui Yang,Danqing Shi

DOI: https://doi.org/10.1007/978-3-030-22602-2_7

2019-01-01

Abstract:The game of Go is a traditional board game originated in ancient China and has long been viewed as the most challenging of board games for artificial intelligence. In 2016, AlphaGo defeated a human professional player in 19 × 19 game with deep neural networks. The ground-breaking advance in computer Go brought artificial intelligence into public view and raised discussions on how to interact with machines in the future. However, AlphaGo is still a highly intelligent computer program without emotion and personality. This paper presents a novel game design of “Naughty AlphaGo”, which is an emotional Go robot player based on AI algorithm. Result shows that the emotion expression approach based on behavioral and kinematic characteristics can support human players’ perception about the AI player’s emotional states through interaction.
DeepMind AI solves geometry problems at star-student level

Davide Castelvecchi

DOI: https://doi.org/10.1038/d41586-024-00141-5

IF: 64.8

2024-01-18

Nature

Abstract:Algorithms are now as good at geometry as some of the world's most mathematically talented school kids. Algorithms are now as good at geometry as some of the world's most mathematically talented school kids.

multidisciplinary sciences
AlphaGomoku: An AlphaGo-based Gomoku Artificial Intelligence using Curriculum Learning

Zheng Xie,XingYu Fu,JinYuan Yu

DOI: https://doi.org/10.48550/arXiv.1809.10595

IF: 14.4

2018-09-27

Artificial Intelligence

Abstract:In this project, we combine AlphaGo algorithm with Curriculum Learning to crack the game of Gomoku. Modifications like Double Networks Mechanism and Winning Value Decay are implemented to solve the intrinsic asymmetry and short-sight of Gomoku. Our final AI AlphaGomoku, through two days' training on a single GPU, has reached humans' playing level.
The Game Is Not over Yet—Go in the Post-AlphaGo Era

Attila Egri-Nagy,Antti Törmänen

DOI: https://doi.org/10.3390/philosophies5040037

2020-11-13

Philosophies

Abstract:The game of Go was the last great challenge for artificial intelligence in abstract board games. AlphaGo was the first system to reach supremacy, and subsequent implementations further improved the state of the art. As in chess, the fall of the human world champion did not lead to the end of the game. Now, we have renewed interest in the game due to new questions that emerged in this development. How far are we from perfect play? Can humans catch up? How compressible is Go knowledge? What is the computational complexity of a perfect player? How much energy is really needed to play the game optimally? Here, we investigate these and related questions with respect to the special properties of Go (meaningful draws and extreme combinatorial complexity). Since traditional board games have an important role in human culture, our analysis is relevant in a broader context. What happens in the game world could forecast our relationship with AI entities, their explainability, and usefulness.
Adversarial Policies Beat Superhuman Go AIs

Tony T. Wang,Adam Gleave,Tom Tseng,Kellin Pelrine,Nora Belrose,Joseph Miller,Michael D. Dennis,Yawen Duan,Viktor Pogrebniak,Sergey Levine,Stuart Russell

2023-07-13

Abstract:We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our adversaries do not win by playing Go well. Instead, they trick KataGo into making serious blunders. Our attack transfers zero-shot to other superhuman Go-playing AIs, and is comprehensible to the extent that human experts can implement it without algorithmic assistance to consistently beat superhuman AIs. The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack. Our results demonstrate that even superhuman AI systems may harbor surprising failure modes. Example games are available <a class="link-external link-https" href="https://goattack.far.ai/" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence,Cryptography and Security

Google AI algorithm masters ancient game of Go

Google reveals secret test of AI bot to beat top Go players

Mastering the game of Go without human knowledge

Mastering the game of Go with deep neural networks and tree search

When Doctors Meet with AlphaGo: Potential Application of Machine Learning to Clinical Medicine.

DeepMind AI is a games master

Human vs. Computer Go: Review and Prospect

[Deep Learning and AlphaGo]

DeepMind AI creates algorithms that sort data faster than those built by people

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

AI cracks hard maths

Teaching Deep Convolutional Neural Networks to Play Go

Very simple statistical evidence that AlphaGo has exceeded human limits in playing GO game

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

AlphaZero Gomoku

Schlau, schlauer, am schlausten: AlphaGo Zero

“Naughty AlphaGo”: Transforming the Game of Computer Go into an Emotional Tangible Playground

DeepMind AI solves geometry problems at star-student level

AlphaGomoku: An AlphaGo-based Gomoku Artificial Intelligence using Curriculum Learning

The Game Is Not over Yet—Go in the Post-AlphaGo Era

Adversarial Policies Beat Superhuman Go AIs