Abstract:Although robots have been widely used in a variety of fields, the idea of enabling them to perform multiple tasks in the same way that humans do remains a difficulty. To solve this, we investigate the learning from demonstration (LFD) system with our independently designed symmetrical humanoid dual-arm robot. We present a novel action feature matching algorithm. This algorithm accurately transforms human demonstration data into task models that robots can directly execute, considerably improving LFD's generalization capabilities. In our studies, we used motion capture cameras to capture human demonstration actions, which included combinations of simple actions (the action layer) and a succession of complicated operational tasks (the task layer). For the action layer data, we employed Gaussian mixture models (GMM) for processing and constructing an action primitive library. As for the task layer data, we created a "keyframe" segmentation method to transform this data into a series of action primitives and build another action primitive library. Guided by our algorithm, the robot successfully imitated complex human tasks. Results show its excellent task learning and execution, providing an effective solution for robots to learn from human demonstrations and significantly advancing robot technology.

What problem does this paper attempt to address?

The paper primarily focuses on addressing the learning and imitation capabilities of robots when performing complex tasks, especially in enabling robots to execute a variety of tasks like humans. To achieve this goal, the research team designed a novel action feature matching algorithm and combined it with their independently developed symmetrical dual-arm humanoid robot for experimentation. This algorithm can precisely transform human demonstration data into task models that robots can directly execute, thereby significantly enhancing the generalization ability of the Learning from Demonstration (LFD) method. Specifically, the work in the paper can be divided into the following key points: 1. **Action Feature Matching Algorithm**: This is an innovative method that can accurately match human demonstration data with the robot's action primitives to form executable task models for the robot. Through this method, robots can better understand and replicate complex human behaviors. 2. **Motion Segmentation Method**: The paper proposes a motion segmentation method based on "keyframes" to reasonably decompose complex operational tasks into a series of action segments. This method helps to transform tasks into a series of action primitives that can be understood by robots. 3. **Action Primitive Library Construction**: Researchers used Gaussian Mixture Models (GMM) to process action-layer data and constructed an action primitive library. This library contains basic actions (such as moving, rotating, etc.), and combinations of these basic actions can represent more complex movements. 4. **Dual-Arm Robot Design and Application**: The dual-arm robot in the paper has a biomimetic structure, capable of flexibly performing various planar movements while ensuring safe human-robot interaction and workspace sharing. Combined with the Robot Operating System (ROS), the robot can directly acquire sensor data and execute movement and manipulation tasks. 5. **Experimental Validation**: Researchers conducted two experiments to demonstrate that the proposed dual-arm robot teaching and learning system can effectively perform various tasks, including but not limited to grasping and manipulating complex objects. This will provide strong support for the application of robots in industries, services, and other fields. In summary, the paper proposes an efficient LFD system aimed at enabling robots to complete self-learning by observing and learning from human demonstrations, thus performing complex operational tasks. This opens up new possibilities for the development of robotic technology.

Research on LFD System of Humanoid Dual-Arm Robot

Learning Robot Manipulation Skills from Human Demonstration Videos Using Two-Stream 2-D/3-D Residual Networks with Self-Attention

COMBINATION OF AFFINE DEFORMATION AND DYNAMIC MOVEMENT PRIMITIVE IN LEARNING HUMAN MOTION FOR REDUNDANT MANIPULATOR

A Learning from Demonstration Method for Robotic Assembly with a Dual-Sub-6-DoF Parallel Robot *

LfDT: Learning Dual-Arm Manipulation from Demonstration Translated from a Human and Robotic Arm

Learning Human Compliant Behavior from Demonstration for Force-Based Robot Manipulation

Fuzzy dynamical system for robot learning motion skills from human demonstration

Diff-LfD: Contact-aware Model-based Learning from Visual Demonstration for Robotic Manipulation Via Differentiable Physics-based Simulation and Rendering.

Learning from Demonstration for 7-DOF Anthropomorphic Manipulators Without Offset Via Analytical Inverse Kinematics

Skill Generalization for Dual-Arm Robot Flipping Manipulation Using OLC-DMP Method

A survey of robot learning from demonstration

Robot Learning from Demonstration for Assembly with Camera-Supplemented Optical Motion Capture Sensors

A Learning from Demonstration Method for Generating Human-like Actions on Redundant Manipulators

Learning from Demonstration Framework for Multi-Robot Systems Using Interaction Keypoints and Soft Actor-Critic Methods

Combining Learning from Demonstration with Learning by Exploration to Facilitate Contact-Rich Tasks

Robot learning from demonstration for path planning: A review

A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations

Human-Robot Skill Transfer with Enhanced Compliance via Dynamic Movement Primitives

Whole-Body Teleoperation Control of Dual-Arm Robot Using Sensor Fusion

A Multitasking-Oriented Robot Arm Motion Planning Scheme Based on Deep Reinforcement Learning and Twin Synchro-Control

Real-Time Whole-Body Imitation by Humanoid Robots and Task-Oriented Teleoperation Using an Analytical Mapping Method and Quantitative Evaluation