Abstract:This paper presents a framework for real-time high-fidelity upper limb motion transfer from a human to a humanoid robot, intending to provide robust support for robots to learn manipulation skills. Robot imitation of human motion is modeled as a simultaneous wrist and elbow tracking problem, where the coordination between multiple imitation objectives and physical constraints receives special attention. First, for the vision-based motion capture system, an efficient preprocessing method is introduced to map the raw human skeletal points analogously or reflectively to the robot’s workspace. Then, the generation of velocity-level imitation motion commands is formalized as a constrained hierarchical quadratic programing (HQP) problem, in which multiple objectives for achieving imitation are organized into a hierarchy. The wrist tracking task is set as the highest priority, followed by the adjustment of the elbow and the minimization of the joint velocity. Physical limits of the robot joints, as well as strict kinematic constraints regarding self-motion, are considered as constraints for the optimization problem. Finally, a hierarchical recurrent neural network (RNN) is designed to solve this HQP problem through its unique neural dynamics. The performance of this network is theoretically guaranteed with high computational efficiency. Quantitative results show that the proposed motion imitation control scheme offers excellent integrated tracking accuracy of [Formula: see text] m and is robust to different arm length ratios. Using this method, the humanoid robot can imitate the upper body motions of a human as accurately as possible under physical constraints, without significant delays.

Motion Similarity Evaluation between Human and a Tri-Co Robot during Real-Time Imitation with a Trajectory Dynamic Time Warping Model

Motion Imitation of a Humanoid Robot Via Pose Estimation

Human-robot Interaction Oriented Human-in-the-loop Real-time Motion Imitation on a Humanoid Tri-Co Robot

HumRoboSim: an Autonomous Humanoid Robot Simulation System

Human Demonstration Trajectory Refinement for Redundant Manipulators.

Virtual-joint Based Motion Similarity Criteria for Human-Robot Kinematics Mapping

Development of a Synchronized Human-Robot-Virtuality Interaction System Using Cooperative Robot and Motion Capture Device

Real-Time Whole-Body Imitation by Humanoid Robots and Task-Oriented Teleoperation Using an Analytical Mapping Method and Quantitative Evaluation

Design and similarity evaluation on humanoid motion based on human motion capture

Whole-body humanoid robot imitation with pose similarity evaluation

Fast and Intuitive Kinematics Mapping for Human-Robot Motion Imitating: A Virtual-Joint-Based Approach

Robust Real-Time Upper Limb Motion Transfer from Human to Humanoid for Manipulation Skill Teaching

A Universal Self-Adaption Workspace Mapping Method for Human–Robot Interaction Using Kinect Sensor Data

Spatiotemporal Mapping and Similarity Evaluation Based on Isomer Motion

A Motion-based User Interface for the Control of Virtual Humans Performing Sports.

Fast Human Whole Body Motion Imitation Algorithm For Humanoid Robots

Humanoid Robot Imitation with Pose Similarity Metric Learning

Real-time Imitation Human Upper-Body Movement with Joint and Self-Collision Constraints

Motion Similarity Comprehensive Evaluation of Humanoid Robot Arm Based on Dynamic Comprehensive Evaluation Method

Interactive Humanoid Robot Arm Imitation System Using Human Upper Limb Motion Tracking

Kinematics Mapping of Humanoid Motion Based on Human Motion