Abstract:Human-robot collaboration (HRC) in a shared workspace has become a common pattern in real-world robot applications and has garnered significant research interest. However, most existing studies for human-in-the-loop (HITL) collaboration with robots in a shared workspace evaluate in either simplified game environments or physical platforms, falling short in limited realistic significance or limited scalability. To support future studies, we build an embodied framework named HumanTHOR, which enables humans to act in the simulation environment through VR devices to support HITL collaborations in a shared workspace. To validate our system, we build a benchmark of everyday tasks and conduct a preliminary user study with two baseline algorithms. The results show that the robot can effectively assist humans in collaboration, demonstrating the significance of HRC. The comparison among different levels of baselines affirms that our system can adequately evaluate robot capabilities and serve as a benchmark for different robot algorithms. The experimental results also indicate that there is still much room in the area and our system can provide a preliminary foundation for future HRC research in a shared workspace. More information about the simulation environment, experiment videos, benchmark descriptions, and additional supplementary materials can be found on the website: <a class="link-external link-https" href="https://sites.google.com/view/humanthor/" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: Research on human - robot collaboration (HRC) in shared workspaces lacks a system and benchmarking platform that can truly simulate human - machine interaction. Existing research is either carried out in simplified game environments or on physical platforms, but these methods either lack practical significance or have limited scalability. Specifically, the paper attempts to solve the following problems: 1. **Limitations of existing systems**: - Research on physical systems has high costs, limited repeatability, and a small scale. - 2D game environments are too simplified and difficult to be generalized to real - physical - robot scenarios. - Although existing simulators can well support the research of robot algorithms, they are insufficient in terms of human - machine interaction, especially lacking support for real - time human - robot collaboration and multi - modal communication. 2. **Key requirements for human - robot collaboration**: - It is required to support **real - time interaction**, that is, humans can participate in experiments in real - time instead of through agents or offline data. - It is required to support collaboration in **shared workspaces**, where humans and robots share the same observation perspectives and capabilities. - It is required to support **instant communication**, including the transmission of multi - modal information such as images and text. - It is required to provide an immersive VR interface to make human activities closer to real - life scenarios. To solve these problems, the author has developed the HumanTHOR system, which is a simulation environment based on the AI2THOR framework, allowing humans to conduct real - time collaboration with robots in shared workspaces through VR devices and providing multi - modal communication functions. In addition, the author has also designed a benchmark test to evaluate the performance of robots in daily tasks, such as object - target navigation and mobile manipulation tasks. In summary, the core objective of this paper is to build a system and benchmarking platform that fully supports human - robot collaboration research, in order to overcome the limitations of existing research methods and lay the foundation for future human - robot collaboration research.

Demonstrating HumanTHOR: A Simulation Platform and Benchmark for Human-Robot Collaboration in a Shared Workspace

A Universal Self-Adaption Workspace Mapping Method for Human–Robot Interaction Using Kinect Sensor Data

Human-Robot Interaction in a Shared Augmented Reality Workspace

CoHRT: A Collaboration System for Human-Robot Teamwork

Enabling collaborative assembly between humans and robots using a digital twin system

A Complementary Framework for Human-Robot Collaboration with a Mixed AR-Haptic Interface

Benchmarking Adaptive Intelligence and Computer Vision on Human-Robot Collaboration

FACT: A Full-body Ad-hoc Collaboration Testbed for Modeling Complex Teamwork

An adaptive human sensor framework for human–robot collaboration

The Biomechanical Evaluation of a Human-Robot Collaborative Task in a Physically Interactive Virtual Reality Simulation Testbed

RoboTHOR: An Open Simulation-to-Real Embodied AI Platform

"Grip-that-there": An Investigation of Explicit and Implicit Task Allocation Techniques for Human-Robot Collaboration

HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation

A Data-driven Approach Towards Human-robot Collaborative Problem Solving in a Shared Space

Workspace Optimization Techniques to Improve Prediction of Human Motion During Human-Robot Collaboration

Human-Robot Gym: Benchmarking Reinforcement Learning in Human-Robot Collaboration

How to be Helpful? Supportive Behaviors and Personalization for Human-Robot Collaboration

Hierarchical Intention Tracking for Robust Human-Robot Collaboration in Industrial Assembly Tasks

Proactive Human-Robot Co-Assembly: Leveraging Human Intention Prediction and Robust Safe Control

How Human-Robot Collaboration Impacts Construction Productivity: an Agent-Based Multi-Fidelity Modeling Approach

Design and Evaluation of an Augmented Reality Head-Mounted Display Interface for Human Robot Teams Collaborating in Physically Shared Manufacturing Tasks