Abstract:This paper presents a framework for real-time high-fidelity upper limb motion transfer from a human to a humanoid robot, intending to provide robust support for robots to learn manipulation skills. Robot imitation of human motion is modeled as a simultaneous wrist and elbow tracking problem, where the coordination between multiple imitation objectives and physical constraints receives special attention. First, for the vision-based motion capture system, an efficient preprocessing method is introduced to map the raw human skeletal points analogously or reflectively to the robot’s workspace. Then, the generation of velocity-level imitation motion commands is formalized as a constrained hierarchical quadratic programing (HQP) problem, in which multiple objectives for achieving imitation are organized into a hierarchy. The wrist tracking task is set as the highest priority, followed by the adjustment of the elbow and the minimization of the joint velocity. Physical limits of the robot joints, as well as strict kinematic constraints regarding self-motion, are considered as constraints for the optimization problem. Finally, a hierarchical recurrent neural network (RNN) is designed to solve this HQP problem through its unique neural dynamics. The performance of this network is theoretically guaranteed with high computational efficiency. Quantitative results show that the proposed motion imitation control scheme offers excellent integrated tracking accuracy of [Formula: see text] m and is robust to different arm length ratios. Using this method, the humanoid robot can imitate the upper body motions of a human as accurately as possible under physical constraints, without significant delays.

Fast and Intuitive Kinematics Mapping for Human-Robot Motion Imitating: A Virtual-Joint-Based Approach

Motion Imitation of a Humanoid Robot Via Pose Estimation

The Design and Implementation of Natural Human-Robot Interaction System Based on Kinect Sensor

HumRoboSim: an Autonomous Humanoid Robot Simulation System

Human-robot Kinematics Mapping Method Based on Dynamic Equivalent Points

Human–robot Kinematic Mapping Method Based on Index Constraint

Robust Real-Time Upper Limb Motion Transfer from Human to Humanoid for Manipulation Skill Teaching

Fast Human Whole Body Motion Imitation Algorithm For Humanoid Robots

Motion Similarity Evaluation between Human and a Tri-Co Robot during Real-Time Imitation with a Trajectory Dynamic Time Warping Model

Inverse Kinematics Analysis of Humanoid Robot Arm by Fusing Denavit–Hartenberg and Screw Theory to Imitate Human Motion With Kinect

Virtual-joint Based Motion Similarity Criteria for Human-Robot Kinematics Mapping

Interactive Humanoid Robot Arm Imitation System Using Human Upper Limb Motion Tracking

A Universal Self-Adaption Workspace Mapping Method for Human–Robot Interaction Using Kinect Sensor Data

Kinematics Mapping of Humanoid Motion Based on Human Motion

A Real-Time Upper-Body Robot Imitation System

Real-time Imitation Human Upper-Body Movement with Joint and Self-Collision Constraints

Real-Time Whole-Body Imitation by Humanoid Robots and Task-Oriented Teleoperation Using an Analytical Mapping Method and Quantitative Evaluation

Human-robot Interaction Oriented Human-in-the-loop Real-time Motion Imitation on a Humanoid Tri-Co Robot

Motion Mapping from a Human Arm to a Heterogeneous Excavator-like Robotic Arm for Intuitive Teleoperation.

Towards human-like kinematics in industrial robotic arms: a case study on a UR3 robot

A Geometric Motion Mapping Algorithm from Master Human Arm to Heterogeneous Slave Robot Arm with Offset Joints