Thespian: Multi-Character Text Role-Playing Game Agents

Christopher Cui,Xiangyu Peng,Mark Riedl

2023-08-04

Abstract:Text-adventure games and text role-playing games are grand challenges for reinforcement learning game playing agents. Text role-playing games are open-ended environments where an agent must faithfully play a particular character. We consider the distinction between characters and actors, where an actor agent has the ability to play multiple characters. We present a framework we call a thespian agent that can learn to emulate multiple characters along with a soft prompt that can be used to direct it as to which character to play at any time. We further describe an attention mechanism that allows the agent to learn new characters that are based on previously learned characters in a few-shot fashion. We show that our agent outperforms the state of the art agent framework in multi-character learning and few-shot learning.

Artificial Intelligence,Computation and Language

What problem does this paper attempt to address?

The paper attempts to address the problem of how to train a reinforcement learning agent in text adventure games and text role-playing games (TRPG) to play multiple roles and quickly learn the behavior patterns of new roles with minimal additional training.

Specifically, the paper focuses on two challenges:
1. **Multi-role Learning**: Training a single reinforcement learning agent model to switch between different role types based on simple instructions. To this end, the authors propose a new RL agent that can simultaneously learn to imitate multiple roles and generate a set of action distributions to represent the behavior choices of each role category. Additionally, the agent learns a soft prompt that can serve as an indicator to tell the agent which specific role to imitate when needed.
2. **Few-shot Learning of New Roles**: Training an agent that can quickly learn new roles while maintaining performance on previously trained roles. By adding an attention mechanism to the agent's output, the agent can learn how to blend the action probabilities of different roles, thereby learning new roles and their new soft prompts. This allows the agent to learn new roles 10 times faster than training from scratch and without degrading the performance of the original roles.

The proposed solution in the paper is the introduction of the "thespian agent" framework, which not only understands and imitates multiple roles but also quickly adapts to new roles with few samples without forgetting previously learned roles. This method has shown superior performance over existing techniques in experiments, particularly in multi-role learning and few-shot learning.