Abstract:Deep Reinforcement Learning algorithms lead to agents that can solve difficult decision making problems in complex environments. However, many difficult multi-agent competitive games, especially real-time strategy games are still considered beyond the capability of current deep reinforcement learning algorithms, although there has been a recent effort to change this\citep {openai_2017_dota, vinyals_2017_starcraft}. Moreover, when the opponents in a competitive game are suboptimal, the current\textit {Nash Equilibrium} seeking, self-play algorithms are often unable to generalize their strategies to opponents that play strategies vastly different from their own. This suggests that a learning algorithm that is beyond conventional self-play is necessary. We develop Hierarchical Agent with Self-play (HASP), a learning approach for obtaining hierarchically structured policies that can achieve higher performance than conventional self-play on competitive games through the use of a diverse pool of sub-policies we get from Counter Self-Play (CSP). We demonstrate that the ensemble policy generated by HASP can achieve better performance while facing unseen opponents that use sub-optimal policies. On a motivating iterated Rock-Paper-Scissor game and a partially observable real-time strategic game (http://generals. io/), we are led to the conclusion that HASP can perform better than conventional self-play as well as achieve 77% win rate against FloBot, an open-source agent which has ranked at position number 2 on the online leaderboards.

Deep Reinforcement Learning for Doom using Unsupervised Auxiliary Tasks

Deep Reinforcement Learning From Raw Pixels in Doom

Playing a FPS Doom Video Game with Deep Visual Reinforcement Learning

Training an Agent for FPS Doom Game Using Visual Reinforcement Learning and VizDoom

Using VizDoom Research Platform Scenarios for Benchmarking Reinforcement Learning Algorithms in First-Person Shooter Games

Playing FPS Games with Deep Reinforcement Learning

Evaluating reinforcement learning algorithms in first-person shooter games using VizDoom

On the Development of an Autonomous Agent for a 3D First-Person Shooter Game Using Deep Reinforcement Learning

Arnold: An Autonomous Agent to Play FPS Games

Reinforcement Learning Applied to AI Bots in First-Person Shooters: A Systematic Review

Combo-Action: Training Agent for FPS Game with Auxiliary Tasks.

Reinforcement Learning with Unsupervised Auxiliary Tasks

Hierarchical Deep Reinforcement Learning Agent with Counter Self-play on Competitive Games

An Improved Approach Towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Playing FPS Games with Environment-Aware Hierarchical Reinforcement Learning

Augmenting Automated Game Testing with Deep Reinforcement Learning

A Human Mixed Strategy Approach to Deep Reinforcement Learning

Optimal Skipping Rates: Training Agents with Fine-Grained Control Using Deep Reinforcement Learning

Research on Game-Playing Agents Based on Deep Reinforcement Learning.

Autoencoder-augmented Neuroevolution for Visual Doom Playing

Design of Artificial Intelligence Agents for Games using Deep Reinforcement Learning