Abstract:A key challenge in human-robot collaboration is the non-stationarity created by humans due to changes in their behaviour. This alters environmental transitions and hinders human-robot collaboration. We propose a principled meta-learning framework to explore how robots could better predict human behaviour, and thereby deal with issues of non-stationarity. On the basis of this framework, we developed Behaviour-Transform (BeTrans). BeTrans is a conditional transformer that enables a robot agent to adapt quickly to new human agents with non-stationary behaviours, due to its notable performance with sequential data. We trained BeTrans on simulated human agents with different systematic biases in collaborative settings. We used an original customisable environment to show that BeTrans effectively collaborates with simulated human agents and adapts faster to non-stationary simulated human agents than SOTA techniques.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the environmental change caused by the non - stationarity of human behavior in human - robot collaboration, which will hinder the effective cooperation between robots and humans. Specifically, the paper focuses on how to enable robots to better predict the behavior of human agents with non - stationary behavior, so as to meet this challenge. To solve this problem, the author proposes a meta - learning - based framework and develops a conditional Transformer model named "Behaviour - Transform (BeTrans)". This model enables robots to quickly adapt to new human agents with non - stationary behavior, and especially performs well in processing sequence data. ### Key issues 1. **Non - stationarity problem**: The non - stationarity of human behavior will lead to changes in environmental dynamics, which makes it difficult for traditional reinforcement learning methods to adapt. 2. **Behavior prediction**: How to enable robots to predict human behavior more accurately, especially when human behavior changes. 3. **Quick adaptation**: How to enable robots to quickly adapt when encountering new non - stationary human behavior. ### Solutions - **Meta - learning framework**: The paper proposes a meta - learning framework, aiming to explore how robots can better predict human behavior, so as to deal with the non - stationarity problem. - **Behaviour - Transform (BeTrans)**: A model based on the Transformer architecture, which can effectively predict and adapt to changes in human behavior in a multi - agent setting. - **Training method**: By training BeTrans in a simulated environment, it can handle human agents with different systematic biases, thus showing better performance in practical applications. ### Experimental verification - **Experimental setup**: The author designed a virtual "batting game" environment to simulate the cooperation task between humans and robots. - **Comparison methods**: Compare with several baseline methods (such as SAC, LILI, RNN - LILI) to verify the effectiveness of BeTrans. - **Experimental results**: The experimental results show that BeTrans can effectively predict human behavior under various conditions (such as fixed behavior, noisy observation, behavior change, long - term dependence), and outperforms existing methods in performance. ### Summary The main contribution of this paper is to propose a new meta - learning framework and a Transformer - based model BeTrans, which can effectively solve the non - stationarity problem in human - robot collaboration and improve the adaptability and cooperation efficiency of robots in dynamic environments.

A behavioural transformer for effective collaboration between a robot and a non-stationary human

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

Enhancing team performance with transfer-learning during real-world human-robot collaboration

Learning on the Job: Long-Term Behavioural Adaptation in Human-Robot Interactions

Adapting to Human Preferences to Lead or Follow in Human-Robot Collaboration: A System Evaluation

Enabling Multi-Robot Collaboration from Single-Human Guidance

Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems

RILI: Robustly Influencing Latent Intent

InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions

Body Transformer: Leveraging Robot Embodiment for Policy Learning

Towards Proactive Safe Human-Robot Collaborations via Data-Efficient Conditional Behavior Prediction

Learning Physical Collaborative Robot Behaviors From Human Demonstrations

Behavior Transformers: Cloning $k$ modes with one stone

Identifying Interaction Patterns of Tangible Co-Adaptations in Human-Robot Team Behaviors

Generalized Behavior Framework for Mobile Robots Teaming With Humans in Harsh Environments

Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interaction

Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind

Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation

Towards Modeling and Influencing the Dynamics of Human Learning

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction

Efficient Model Learning for Human-Robot Collaborative Tasks