Abstract:Recent advances in Deep Reinforcement Learning (DRL) have largely focused on improving the performance of agents with the aim of replacing humans in known and well-defined environments. The use of these techniques as a game design tool for video game production, where the aim is instead to create Non-Player Character (NPC) behaviors, has received relatively little attention until recently. Turn-based strategy games like Roguelikes, for example, present unique challenges to DRL. In particular, the categorical nature of their complex game state, composed of many entities with different attributes, requires agents able to learn how to compare and prioritize these entities. Moreover, this complexity often leads to agents that overfit to states seen during training and that are unable to generalize in the face of design changes made during development. In this paper we propose two network architectures which, when combined with a \emph{procedural loot generation} system, are able to better handle complex categorical state spaces and to mitigate the need for retraining forced by design decisions. The first is based on a dense embedding of the categorical input space that abstracts the discrete observation model and renders trained agents more able to generalize. The second proposed architecture is more general and is based on a Transformer network able to reason relationally about input and input attributes. Our experimental evaluation demonstrates that new agents have better adaptation capacity with respect to a baseline architecture, making this framework more robust to dynamic gameplay changes during development. Based on the results shown in this paper, we believe that these solutions represent a step forward towards making DRL more accessible to the gaming industry.

Reinforcement learning for game personalization on edge devices

Reinforcement Learning in Game Industry—Review, Prospects and Challenges

Interactive Narrative Personalization with Deep Reinforcement Learning

FDQN: A Flexible Deep Q-Network Framework for Game Automation

State of the Art Control of Atari Games Using Shallow Reinforcement Learning

EdgeRL: Reinforcement Learning-driven Deep Learning Model Inference Optimization at Edge

Towards scalable and efficient Deep-RL in edge computing: A game-based partition approach

Architecting and Visualizing Deep Reinforcement Learning Models

Accelerating Reinforcement Learning using EEG-based implicit human feedback

Deep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike Games

Pixel to policy: DQN Encoders for within & cross-game reinforcement learning

Beyond The Rainbow: High Performance Deep Reinforcement Learning On A Desktop PC

A Survey of Deep Reinforcement Learning in Video Games

Beyond the Edge: An Advanced Exploration of Reinforcement Learning for Mobile Edge Computing, its Applications, and Future Research Trajectories

Learning to Play Text-based Adventure Games with Maximum Entropy Reinforcement Learning

Generating intelligent agent behaviors in multi-agent game AI using deep reinforcement learning algorithm

Reinforcement Learning Agents for Ubisoft's Roller Champions

Novel Reinforcement Learning Research Platform for Role-Playing Games

Deep reinforcement learning from human preferences

Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling

Reinforcement Learning and Video Games