Abstract:In this paper, we propose and validate a Joint-Initiative Supervised Autonomy (JISA) framework for Human-Robot Interaction (HRI), in which a robot maintains a measure of its self-confidence (SC) while performing a task, and only prompts the human supervisor for help when its SC drops. At the same time, during task execution, a human supervisor can intervene in the task being performed, based on his/her Situation Awareness (SA). To evaluate the applicability and utility of JISA, it is implemented on two different HRI tasks: grid-based collaborative simultaneous localization and mapping (SLAM) and automated jigsaw puzzle reconstruction. Augmented Reality (AR) (for SLAM) and two-dimensional graphical user interfaces (GUI) (for puzzle reconstruction) are custom-designed to enhance human SA and allow intuitive interaction between the human and the agent. The superiority of the JISA framework is demonstrated in experiments. In SLAM, the superior maps produced by JISA preclude the need for post processing of any SLAM stock maps; furthermore, JISA reduces the required mapping time by approximately 50 percent versus traditional approaches. In automated puzzle reconstruction, the JISA framework outperforms both fully autonomous solutions, as well as those resulting from on-demand human intervention prompted by the agent.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively utilize human supervision in human - robot interaction (HRI) to improve the performance of robots when performing tasks, while reducing errors in fully autonomous systems. Specifically, the paper proposes a framework named Joint - Initiative Supervised Autonomy (JISA), aiming to improve the performance of automated tasks and alleviate the limitations of fully autonomous systems without requiring humans to completely take over the tasks by combining the robot's Self - Confidence (SC) and human's Situation Awareness (SA). The paper points out that with the development of robotics, robots are being applied in increasingly complex and unstructured environments, which pose challenges to fully autonomous robots, such as limitations in perception, cognition, and execution capabilities. These problems may lead to errors when robots perform tasks. In addition, when robots are in a fully autonomous state, the noise in the real world increases the probability of making mistakes. These errors may be due to the uncertainty of certain decisions or situations, or due to limitations in software/hardware capabilities. In this case, the robot may be aware of potential problems (for example, high uncertainty in object detection or inability to reach the operation target), or may not be aware of the existence of errors (for example, LiDAR fails to detect transparent obstacles). To solve the above problems, the paper proposes the JISA framework. This framework allows the robot to maintain a certain level of self - confidence when performing tasks and only asks for help from human supervisors when its self - confidence drops. Meanwhile, during the task execution, human supervisors can intervene in the ongoing task according to their own situation awareness. To evaluate the applicability and practicality of the JISA framework, the researchers implemented JISA in two different HRI tasks: grid - based collaborative Simultaneous Localization and Mapping (SLAM) and automatic jigsaw puzzle reconstruction. Through custom - designed augmented reality (AR) (for SLAM) and two - dimensional graphical user interface (GUI) (for jigsaw puzzle reconstruction), human situation awareness is enhanced, and intuitive interaction between humans and agents is allowed. The experimental results show that the JISA framework generates maps of higher quality in SLAM without the need for post - processing of any SLAM standard maps; and compared with traditional methods, JISA reduces the mapping time by about 50%. In automatic jigsaw puzzle reconstruction, the performance of the JISA framework is better than that of fully autonomous solutions and the results of human interventions triggered by agents.

Human-Robot Interaction via a Joint-Initiative Supervised Autonomy (JISA) Framework

Development of a Human-Robot Hybrid Intelligent System Based on Brain Teleoperation and Deep Learning SLAM

Design of Kinect-Based Human Robot Interaction Systems for A Robocup Middle Size League Soccer Robot

A Universal Self-Adaption Workspace Mapping Method for Human–Robot Interaction Using Kinect Sensor Data

Collaborative Autonomy: Human–Robot Interaction to the Test of Intelligent Help

Human-robot interaction through adjustable social autonomy

Evaluating the Augmented Reality Human-Robot Collaboration System

Design and Implementation of a Human-Robot Joint Action Framework using Augmented Reality and Eye Gaze

A Complementary Framework for Human-Robot Collaboration with a Mixed AR-Haptic Interface

Study of Human–Robot Interactions for Assistive Robots Using Machine Learning and Sensor Fusion Technologies

Human-Robot Interaction in a Shared Augmented Reality Workspace

An Augmented Reality Human-Robot Collaboration System

Implementation of Engagement Detection for Human–Robot Interaction in Complex Environments

An AR-assisted Deep Reinforcement Learning-based approach towards mutual-cognitive safe human-robot interaction

A framework for trust-related knowledge transfer in human–robot interaction

Human–robot interaction via voice-controllable intelligent user interface

Human-Robot Interaction using VAHR: Virtual Assistant, Human, and Robots in the Loop

Interaction Modeling and Classification Scheme for Augmenting the Response Accuracy of Human-Robot Interaction Systems.

Formalizing and Guaranteeing* Human-Robot Interaction

Integration of Tracking, Re-Identification, and Gesture Recognition for Facilitating Human-Robot Interaction

A Human Motion Compensation Framework for a Supernumerary Robotic Arm