Abstract:Recent advances in Deep Reinforcement Learning (DRL) have largely focused on improving the performance of agents with the aim of replacing humans in known and well-defined environments. The use of these techniques as a game design tool for video game production, where the aim is instead to create Non-Player Character (NPC) behaviors, has received relatively little attention until recently. Turn-based strategy games like Roguelikes, for example, present unique challenges to DRL. In particular, the categorical nature of their complex game state, composed of many entities with different attributes, requires agents able to learn how to compare and prioritize these entities. Moreover, this complexity often leads to agents that overfit to states seen during training and that are unable to generalize in the face of design changes made during development. In this paper we propose two network architectures which, when combined with a \emph{procedural loot generation} system, are able to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions. The first is based on a dense embedding of the categorical input space that abstracts the discrete observation model and renders trained agents more able to generalize. The second proposed architecture is more general and is based on a Transformer network able to reason relationally about input and input attributes. Our experimental evaluation demonstrates that new agents have better adaptation capacity with respect to a baseline architecture, making this framework more robust to dynamic gameplay changes during development. Based on the results shown in this paper, we believe that these solutions represent a step forward towards making DRL more accessible to the gaming industry.

A Modular Deep-learning Environment for Rogue

Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games

Novel Reinforcement Learning Research Platform for Role-Playing Games

The NetHack Learning Environment

Deep Reinforcement Learning From Raw Pixels in Doom

Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design

DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

Deep Apprenticeship Learning for Playing Games

Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban

Modular Architecture for StarCraft II with Deep Reinforcement Learning

Playing NetHack with LLMs: Potential & Limitations as Zero-Shot Agents

Deep Reinforcement Learning for Doom using Unsupervised Auxiliary Tasks

Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning

Towards a Deep Reinforcement Learning Approach for Tower Line Wars

Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning

MDP Playground: An Analysis and Debug Testbed for Reinforcement Learning

Godot Reinforcement Learning Agents

Generating intelligent agent behaviors in multi-agent game AI using deep reinforcement learning algorithm

Google Research Football: A Novel Reinforcement Learning Environment

Room Clearance with Feudal Hierarchical Reinforcement Learning