Abstract:Recent advances in Deep Reinforcement Learning (DRL) have largely focused on improving the performance of agents with the aim of replacing humans in known and well-defined environments. The use of these techniques as a game design tool for video game production, where the aim is instead to create Non-Player Character (NPC) behaviors, has received relatively little attention until recently. Turn-based strategy games like Roguelikes, for example, present unique challenges to DRL. In particular, the categorical nature of their complex game state, composed of many entities with different attributes, requires agents able to learn how to compare and prioritize these entities. Moreover, this complexity often leads to agents that overfit to states seen during training and that are unable to generalize in the face of design changes made during development. In this paper we propose two network architectures which, when combined with a \emph{procedural loot generation} system, are able to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions. The first is based on a dense embedding of the categorical input space that abstracts the discrete observation model and renders trained agents more able to generalize. The second proposed architecture is more general and is based on a Transformer network able to reason relationally about input and input attributes. Our experimental evaluation demonstrates that new agents have better adaptation capacity with respect to a baseline architecture, making this framework more robust to dynamic gameplay changes during development. Based on the results shown in this paper, we believe that these solutions represent a step forward towards making DRL more accessible to the gaming industry.

High-Fidelity Simulated Players for Interactive Narrative Planning

Interactive Narrative Personalization with Deep Reinforcement Learning

Generative Personas That Behave and Experience Like Humans

SIMS: Simulating Human-Scene Interactions with Real World Script Planning

Imagined Potential Games: A Framework for Simulating, Learning and Evaluating Interactive Behaviors

Informing a BDI Player Model for an Interactive Narrative

NarrativePlay: Interactive Narrative Understanding

Simulation and Retargeting of Complex Multi-Character Interactions

PlayBest: Professional Basketball Player Behavior Synthesis via Planning with Diffusion

Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games

Player Behavior Modeling for Enhancing Role-Playing Game Engagement

InterSim: Interactive Traffic Simulation Via Explicit Relation Modeling

Learning Interactive Real-World Simulators

StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planning

Towards Interactive Training of Non-Player Characters in Video Games

Simulating Student Interactions with Two-stage Imitation Learning for Intelligent Educational Systems

Promptable Game Models: Text-guided Game Simulation via Masked Diffusion Models

Exploring Gameplay With AI Agents

SurrealDriver: Designing Generative Driver Agent Simulation Framework in Urban Contexts based on Large Language Model

Solving Motion Planning Tasks with a Scalable Generative Model

Intelligent Decision-Making and Human Language Communication Based on Deep Reinforcement Learning in a Wargame Environment