Oriol Vinyals,Igor Babuschkin,Wojciech M. Czarnecki,Michaël Mathieu,Andrew Dudzik,Junyoung Chung,David H. Choi,Richard Powell,Timo Ewalds,Petko Georgiev,Junhyuk Oh,Dan Horgan,Manuel Kroiss,Ivo Danihelka,Aja Huang,Laurent Sifre,Trevor Cai,John P. Agapiou,Max Jaderberg,Alexander S. Vezhnevets,Rémi Leblond,Tobias Pohlen,Valentin Dalibard,David Budden,Yury Sulsky,James Molloy,Tom L. Paine,Caglar Gulcehre,Ziyu Wang,Tobias Pfaff,Yuhuai Wu,Roman Ring,Dani Yogatama,Dario Wünsch,Katrina McKinney,Oliver Smith,Tom Schaul,Timothy Lillicrap,Koray Kavukcuoglu,Demis Hassabis,Chris Apps,David Silver

Abstract:Many real-world applications require artificial agents to compete and coordinate with other agents in complex environments. As a stepping stone to this goal, the domain of StarCraft has emerged as an important challenge for artificial intelligence research, owing to its iconic and enduring status among the most difficult professional esports and its relevance to the real world in terms of its raw complexity and multi-agent challenges. Over the course of a decade and numerous competitions<a href="#ref-CR1">1</a>,<a href="#ref-CR2">2</a>,<a href="/articles/s41586-019-1724-z#ref-CR3">3</a>, the strongest agents have simplified important aspects of the game, utilized superhuman capabilities, or employed hand-crafted sub-systems<a href="/articles/s41586-019-1724-z#ref-CR4">4</a>. Despite these advantages, no previous agent has come close to matching the overall skill of top StarCraft players. We chose to address the challenge of StarCraft using general-purpose learning methods that are in principle applicable to other complex domains: a multi-agent reinforcement learning algorithm that uses data from both human and agent games within a diverse league of continually adapting strategies and counter-strategies, each represented by deep neural networks<a href="/articles/s41586-019-1724-z#ref-CR5">5</a>,<a href="/articles/s41586-019-1724-z#ref-CR6">6</a>. We evaluated our agent, AlphaStar, in the full game of StarCraft II, through a series of online games against human players. AlphaStar was rated at Grandmaster level for all three StarCraft races and above 99.8% of officially ranked human players.

Counter-Strike Deathmatch with Large-Scale Behavioural Cloning

Learning to Move Like Professional Counter‐Strike Players

Benchmarking End-to-End Behavioural Cloning on Video Games

Learning to Move Like Professional Counter-Strike Players

Playing FPS Games with Deep Reinforcement Learning

Arnold: An Autonomous Agent to Play FPS Games

Playing Minecraft with Behavioural Cloning

AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning

Hierarchical Deep Reinforcement Learning Agent with Counter Self-play on Competitive Games

Learning Macromanagement in StarCraft from Replays using Deep Learning

On Multi-Agent Learning in Team Sports Games

Reinforcement Learning Applied to AI Bots in First-Person Shooters: A Systematic Review

Adaptive Shooting for Bots in First Person Shooter Games Using Reinforcement Learning

Behavioural Cloning in VizDoom

Training Interactive Agent in Large FPS Game Map with Rule-enhanced Reinforcement Learning

Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning

Learning to Play by Imitating Humans

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Towards Interactive Training of Non-Player Characters in Video Games

Behavioral Cloning from Observation

Generating intelligent agent behaviors in multi-agent game AI using deep reinforcement learning algorithm