Oriol Vinyals,Timo Ewalds,Sergey Bartunov,Petko Georgiev,Alexander Sasha Vezhnevets,Michelle Yeo,Alireza Makhzani,Heinrich Küttler,John Agapiou,Julian Schrittwieser,John Quan,Stephen Gaffney,Stig Petersen,Karen Simonyan,Tom Schaul,Hado van Hasselt,David Silver,Timothy Lillicrap,Kevin Calderone,Paul Keet,Anthony Brunasso,David Lawrence,Anders Ekermo,Jacob Repp,Rodney Tsing

Abstract:This paper introduces SC2LE (StarCraft II Learning Environment), a reinforcement learning environment based on the StarCraft II game. This domain poses a new grand challenge for reinforcement learning, representing a more difficult class of problems than considered in most prior work. It is a multi-agent problem with multiple players interacting; there is imperfect information due to a partially observed map; it has a large action space involving the selection and control of hundreds of units; it has a large state space that must be observed solely from raw input feature planes; and it has delayed credit assignment requiring long-term strategies over thousands of steps. We describe the observation, action, and reward specification for the StarCraft II domain and provide an open source Python-based interface for communicating with the game engine. In addition to the main game maps, we provide a suite of mini-games focusing on different elements of StarCraft II gameplay. For the main game maps, we also provide an accompanying dataset of game replay data from human expert players. We give initial baseline results for neural networks trained from this data to predict game outcomes and player actions. Finally, we present initial baseline results for canonical deep reinforcement learning agents applied to the StarCraft II domain. On the mini-games, these agents learn to achieve a level of play that is comparable to a novice player. However, when trained on the main game, these agents are unable to make significant progress. Thus, SC2LE offers a new and challenging environment for exploring deep reinforcement learning algorithms and architectures.

Dota 2 with Large Scale Deep Reinforcement Learning

Towards Playing Full MOBA Games with Deep Reinforcement Learning

Beating the World's Best at Super Smash Bros. with Deep Reinforcement Learning

Long-Term Planning and Situational Awareness in OpenAI Five

Hierarchical Deep Reinforcement Learning Agent with Counter Self-play on Competitive Games

Multi-Robot Real-time Game Strategy Learning Based on Deep Reinforcement Learning.

AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning

Deep Reinforcement Learning for General Video Game AI

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

Towards a Deep Reinforcement Learning Approach for Tower Line Wars

TiKick: Toward Playing Multi-agent Football Full Games from Single-agent Demonstrations

A Survey of Deep Reinforcement Learning in Video Games

Mastering Complex Control in MOBA Games with Deep Reinforcement Learning

Playing Card-Based RTS Games with Deep Reinforcement Learning.

Grandmaster level in StarCraft II using multi-agent reinforcement learning

Playing Tetris with Reinforcement Learning

Deep Q-Network for AI Soccer

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

Design of Artificial Intelligence Agents for Games using Deep Reinforcement Learning

SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II

StarCraft II: A New Challenge for Reinforcement Learning