Abstract:Many tasks performed by two humans require mutual interaction between arms such as handing-over tools and objects. In order for a robotic arm to interact with a human in the same way, it must reason about the location of the human arm in real-time. Furthermore and to acquire interaction in a timely manner, the robot must be able predict the final target of the human in order to plan and initiate motion beforehand. In this paper, we explore the use of a low-cost wearable device equipped with two inertial measurement units (IMU) for learning reaching motion for real-time applications of Human-Robot Collaboration (HRC). A wearable device can replace or be complementary to visual perception in cases of bad lighting or occlusions in a cluttered environment. We first train a neural-network model to estimate the current location of the arm. Then, we propose a novel model based on a recurrent neural-network to predict the future target of the human arm during motion in real-time. Early prediction of the target grants the robot with sufficient time to plan and initiate motion during the motion of the human. The accuracies of the models are analyzed concerning the features included in the motion representation. Through experiments and real demonstrations with a robotic arm, we show that sufficient accuracy is achieved for feasible HRC without any visual perception. Once trained, the system can be deployed in various spaces with no additional effort. The models exhibit high accuracy for various initial poses of the human arm. Moreover, the trained models are shown to provide high success rates with additional human participants not included in the model training.

Human-Robot Sign Language Motion Retargeting from Videos

Dynamic Movement Primitive Based Motion Retargeting for Dual-Arm Sign Language Motions

Frame-by-frame Motion Retargeting with Self-Collision Avoidance from Diverse Human Demonstrations

Robust Real-Time Motion Retargeting Via Neural Latent Prediction

Kinematic Motion Retargeting via Neural Latent Optimization for Learning Sign Language

Motion Imitation of a Humanoid Robot Via Pose Estimation

Hand Posture Recognition and Tracking Based on Bag-of-Words for Human Robot Interaction

ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space

A New Method for Motion Retargeting for the Hand Gesture

Research on the improved gesture tracking algorithm in sign language synthesis

Unsupervised Motion Retargeting for Human-Robot Imitation

A Novel Gesture Recognition System for Intelligent Interaction with a Nursing-Care Assistant Robot

A Sensor-Based Method to Capture Motion of Two Arms

Learning Character-Agnostic Motion for Motion Retargeting in 2D

Neural Sign Reenactor: Deep Photorealistic Sign Language Retargeting

A multimodal human-robot sign language interaction framework applied in social robots

Signs of Language: Embodied Sign Language Fingerspelling Acquisition from Demonstrations for Human-Robot Interaction

Spatio-Temporal Motion Retargeting for Quadruped Robots

Semantics-aware Motion Retargeting with Vision-Language Models

Learning Human-arm Reaching Motion Using IMU in Human-Robot Collaboration

ACE: Adversarial Correspondence Embedding for Cross Morphology Motion Retargeting from Human to Nonhuman Characters