Abstract:Recent advances in Deep Reinforcement Learning (DRL) have largely focused on improving the performance of agents with the aim of replacing humans in known and well-defined environments. The use of these techniques as a game design tool for video game production, where the aim is instead to create Non-Player Character (NPC) behaviors, has received relatively little attention until recently. Turn-based strategy games like Roguelikes, for example, present unique challenges to DRL. In particular, the categorical nature of their complex game state, composed of many entities with different attributes, requires agents able to learn how to compare and prioritize these entities. Moreover, this complexity often leads to agents that overfit to states seen during training and that are unable to generalize in the face of design changes made during development. In this paper we propose two network architectures which, when combined with a \emph{procedural loot generation} system, are able to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions. The first is based on a dense embedding of the categorical input space that abstracts the discrete observation model and renders trained agents more able to generalize. The second proposed architecture is more general and is based on a Transformer network able to reason relationally about input and input attributes. Our experimental evaluation demonstrates that new agents have better adaptation capacity with respect to a baseline architecture, making this framework more robust to dynamic gameplay changes during development. Based on the results shown in this paper, we believe that these solutions represent a step forward towards making DRL more accessible to the gaming industry.

RL-DOT: A Reinforcement Learning NPC Team for Playing Domination Games

Grounded Reinforcement Learning: Learning to Win the Game under Human Commands

Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games

Curriculum-RL Based Air Combat Decision-Making

Commander-Soldiers Reinforcement Learning for Cooperative Multi-Agent Systems

AlphaStar: an Integrated Application of Reinforcement Learning Algorithms

Hierarchical Deep Reinforcement Learning Agent with Counter Self-play on Competitive Games

Hierarchical Reinforcement Learning for Multi-agent MOBA Game

Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer

A Deep Reinforcement Learning-Based Method Applied for Solving Multi-Agent Defense and Attack Problems.

Generating intelligent agent behaviors in multi-agent game AI using deep reinforcement learning algorithm

Reinforcement Learning Agents for Ubisoft's Roller Champions

System Design for an Integrated Lifelong Reinforcement Learning Agent for Real-Time Strategy Games

Multi-Robot Real-time Game Strategy Learning Based on Deep Reinforcement Learning.

Air-Combat Strategy Using Deep Q-Learning

Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment

DRE-Bot: A Hierarchical First Person Shooter Bot Using Multiple Sarsa(λ) Reinforcement Learners

Self-Driving Car Racing: Application of Deep Reinforcement Learning

Prioritized League Reinforcement Learning for Large-Scale Heterogeneous Multiagent Systems

Confrontation and Obstacle-Avoidance of Unmanned Vehicles Based on Progressive Reinforcement Learning.

Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning