Abstract:The industrial robot plays an increasingly important role in manufacturing industry, but it is limited by its high prerequisite of programming on users, which is time consuming and challenging. In this context, a mechanism enabling autoprogram of robot is desired. Learning from demonstration system (LDS) is one of systems that aim on this goal. In this paper, we present a graph based representation of knowledge called assembly graph (AG) to describe the knowledge on parts assembly, which is independent on specific workspace or robot. Based on this graph representation, a LDS framework is then proposed for parts assembly to enable knowledge transform and knowledge transfer. The former means that the knowledge on part assembly can be transformed from the human to the robot and the latter means that the knowledge can be transferred from the leaned computer to any other robots with other workspace. Besides, we present an implementation of this conceptual framework consisting of two stages including demonstration and execution. In demonstration stage, a human teacher shows the process of parts assembly, which is recorded by a camera system. The assembly relations are detected from images and represented by AG. It is solved by taking property of parts as well as the robot into consideration to obtain the precise pose of each part. By using the solution, the robot can assemble the parts as shown in demonstration in the execution stage. In the experiment, the building blocks are used for assembly. Our robot succeeds in assembling some building blocks together which is initially demonstrated by a human teacher.

Real-Time Learning and Recognition of Assembly Activities Based on Virtual Reality Demonstration

Towards Learning from Demonstration System for Parts Assembly: A Graph Based Representation for Knowledge

Perception of Demonstration for Automatic Programing of Robotic Assembly: Framework, Algorithm, and Validation

AVR-AKG: using virtual reality for domain knowledge generation from first-person demonstration

A Novel Robot Teaching System Based on Augmented Reality

Extended Reality System for Robotic Learning from Human Demonstration

A Robot 3C Assembly Skill Learning Method by Intuitive Human Assembly Demonstration

Teaching robots to do object assembly using multi-modal 3D vision

Demo: I Am a Robot: First-person Robotic Arm Control with Hand Motions in Virtual Reality.

Application of Virtual Reality Technology in Teaching and Training System of Processing Industrial Robot

Robot-enabled Tangible Virtual Assembly with Coordinated Midair Object Placement

A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations

Learning from demonstrations: An intuitive VR environment for imitation learning of construction robots

Motion capture and AR based programming by demonstration for industrial robots using handheld teaching device

Augmented Reality Demonstrations for Scalable Robot Imitation Learning

One-shot sim-to-real transfer policy for robotic assembly via reinforcement learning with visual demonstration

Design and Implementation of Virtual Teaching System for Humanoid Robot

A Human–Robot Collaboration Method Using a Pose Estimation Network for Robot Learning of Assembly Manipulation Trajectories From Demonstration Videos

Knowledge-Driven Robot Program Synthesis from Human VR Demonstrations

Mid-Air Gestures for Multi-Fingered Virtual Assembly System with Leap Motion.

Virtual Reality Human-machine Interface in Robot Off-line Programming