Abstract:Reinforcement learning simulation environments pose an important experimental test bed and facilitate data collection for developing AI-based robot applications. Most of them, however, focus on single-agent tasks, which limits their application to the development of social agents. This study proposes the Chef's Hat simulation environment, which implements a multi-agent competitive card game that is a complete reproduction of the homonymous board game, designed to provoke competitive strategies in humans and emotional responses. The game was shown to be ideal for developing personalized reinforcement learning, in an online learning closed-loop scenario, as its state representation is extremely dynamic and directly related to each of the opponent's actions. To adapt current reinforcement learning agents to this scenario, we also developed the COmPetitive Prioritized Experience Replay (COPPER) algorithm. With the help of COPPER and the Chef's Hat simulation environment, we evaluated the following: (1) 12 experimental learning agents, trained via four different regimens (self-play, play against a naive baseline, PER, or COPPER) with three algorithms based on different state-of-the-art learning paradigms (PPO, DQN, and ACER), and two "dummy" baseline agents that take random actions, (2) the performance difference between COPPER and PER agents trained using the PPO algorithm and playing against different agents (PPO, DQN, and ACER) or all DQN agents, and (3) human performance when playing against two different collections of agents. Our experiments demonstrate that COPPER helps agents learn to adapt to different types of opponents, improving the performance when compared to off-line learning models. An additional contribution of the study is the formalization of the Chef's Hat competitive game and the implementation of the Chef's Hat Player Club, a collection of trained and assessed agents as an enabler for embedding human competitive strategies in social continual and competitive reinforcement learning.

The Chef's Hat Simulation Environment for Reinforcement-Learning-Based Agents

You Were Always on My Mind: Introducing Chef's Hat and COPPER for Personalized Reinforcement Learning

It's Food Fight! Introducing the Chef's Hat Card Game for Affective-Aware HRI

VRKitchen: an Interactive 3D Environment for Learning Real Life Cooking Tasks

The Ingredients of Real-World Robotic Reinforcement Learning

VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning

Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in First-person Simulated 3D Environments

Real-World Human-Robot Collaborative Reinforcement Learning

Incorporating Rivalry in Reinforcement Learning for a Competitive Game

Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning

Machine Learning of Social States and Skills for Multi-Party Human-Robot Interaction

Machine Learning for Social Multiparty Human-Robot Interaction

Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation

Habitat 2.0: Training Home Assistants to Rearrange their Habitat

Development of a Simulation Environment for Robot Soccer Game with Deep Reinforcement Learning and Role Assignment

Learning Socially Appropriate Robot Approaching Behavior Toward Groups using Deep Reinforcement Learning

Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learning

Exploring Human-robot Interaction by Simulating Robots

Extended Reality System for Robotic Learning from Human Demonstration

Benchmarking Reinforcement Learning Methods for Dexterous Robotic Manipulation with a Three-Fingered Gripper