Abstract:In human–robot shared manufacturing contexts, product parts or tools hand-over between the robot and the human is an important collaborative task. Facilitating the robot to figure out and predict human hand-over intentions correctly to improve the task efficiency in human–robot collaboration is therefore a necessary issue to be addressed. In this study, a teaching-learning-prediction (TLP) framework is proposed for the robot to learn from its human partner's multimodal demonstrations and predict human hand-over intentions. In this approach, the robot can be programmed by the human through demonstrations utilizing natural language and wearable sensors according to task requirements and the human's working preferences. Then the robot learns from human hand-over demonstrations online via extreme learning machine (ELM) algorithms to update its cognition capacity, allowing the robot to use its learned policy to predict human intentions actively and assist its human companion in hand-over tasks. Experimental results and evaluations suggest that the human may program the robot easily by the proposed approach when the task changes, as the robot can effectively predict hand-over intentions with competitive accuracy to complete the hand-over tasks. Note to Practitioners—This article is motivated by human–robot hand-over problems in smart manufacturing contexts. Product parts or tools delivery in worker–robot partnerships is an important collaborative task. We develop a teaching-learning-prediction (TLP) framework for the robot to learn from its human partner's multimodal demonstrations and predict human hand-over intentions. The robot can be taught by human through natural language and wearable sensing information. The extreme learning machine (ELM) approach is employed for the robot to build its cognition ca-acity to predict human intentions actively and assist its human companion in hand-over tasks. We demonstrate that the proposed approach presents distinct and effective advantages to facilitate human–robot hand-over tasks in collaborative manufacturing contexts.

Human-Robot handover task intention recognition framework by fusing human digital twin and deep domain adaptation

A Human-Robot Collaboration System for Object Handover

An Attention-Based Deep Learning Approach for Inertial Motion Recognition and Estimation in Human-Robot Collaboration

A Vision-based Human Digital Twin Modelling Approach for Adaptive Human-Robot Collaboration

Construction of Human Digital Twin Model Based on Multimodal Data and Its Application in Locomotion Mode Identification

Predicting Human Intentions in Human-Robot Hand-Over Tasks Through Multimodal Learning

Learning Human-to-Robot Dexterous Handovers for Anthropomorphic Hand

Robot-To-Human Handover with Obstacle Avoidance Via Continuous Time Recurrent Neural Network

Dynamic Hand Gesture-Featured Human Motor Adaptation in Tool Delivery using Voice Recognition

Human Digital Twin in the Context of Industry 5.0

A Universal Self-Adaption Workspace Mapping Method for Human–Robot Interaction Using Kinect Sensor Data

Multi-Robot Behavior Adaptation to Humans' Intention in Human-Robot Interaction Using Information-Driven Fuzzy Friend-Q Learning.

A Facial Expression Emotion Recognition Based Human-robot Interaction System

A framework and method for Human-Robot cooperative safe control based on digital twin

Robot recognizing humans intention and interacting with humans based on a multi-task model combining ST-GCN-LSTM model and YOLO model

Real-Time Multi-modal Human-Robot Collaboration Using Gestures and Speech

GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation, Demonstration, and Imitation

An adaptive human sensor framework for human–robot collaboration

Human-to-Robot Handover Based on Reinforcement Learning

Cross-View Human Intention Recognition for Human-Robot Collaboration

A Complementary Framework for Human-Robot Collaboration with a Mixed AR-Haptic Interface