Abstract:Recent advances in Deep Reinforcement Learning (DRL) have largely focused on improving the performance of agents with the aim of replacing humans in known and well-defined environments. The use of these techniques as a game design tool for video game production, where the aim is instead to create Non-Player Character (NPC) behaviors, has received relatively little attention until recently. Turn-based strategy games like Roguelikes, for example, present unique challenges to DRL. In particular, the categorical nature of their complex game state, composed of many entities with different attributes, requires agents able to learn how to compare and prioritize these entities. Moreover, this complexity often leads to agents that overfit to states seen during training and that are unable to generalize in the face of design changes made during development. In this paper we propose two network architectures which, when combined with a \emph{procedural loot generation} system, are able to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions. The first is based on a dense embedding of the categorical input space that abstracts the discrete observation model and renders trained agents more able to generalize. The second proposed architecture is more general and is based on a Transformer network able to reason relationally about input and input attributes. Our experimental evaluation demonstrates that new agents have better adaptation capacity with respect to a baseline architecture, making this framework more robust to dynamic gameplay changes during development. Based on the results shown in this paper, we believe that these solutions represent a step forward towards making DRL more accessible to the gaming industry.

G-PCGRL: Procedural Graph Data Generation via Reinforcement Learning

PCGRL: Procedural Content Generation via Reinforcement Learning

PCGRL+: Scaling, Control and Generalization in Reinforcement Learning Level Generators

Adversarial Reinforcement Learning for Procedural Content Generation

Procedural Content Generation via Machine Learning (PCGML)

Graph learning-based generation of abstractions for reinforcement learning

Procedural Content Generation for Game Props? A Study on the Effects on User Experience

Finding path and cycle counting formulae in graphs with Deep Reinforcement Learning

Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art

Feudal Graph Reinforcement Learning

Procedural Content Generation via Generative Artificial Intelligence

Graph Reinforcement Learning for Power Grids: A Comprehensive Survey

PCGPT: Procedural Content Generation via Transformers

GRL: Knowledge graph completion with GAN-based reinforcement learning

Explore with Dynamic Map: Graph Structured Reinforcement Learning

RLgraph: Modular Computation Graphs for Deep Reinforcement Learning

Learning to Generate Levels From Nothing

GRL-PS: Graph Embedding-Based DRL Approach for Adaptive Path Selection

Deep Learning for Procedural Content Generation

Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games

Learning-Based Procedural Content Generation