Abstract:Purpose - The purpose of this paper is to propose a new method based on three-dimensional (3D) vision technologies and human skill integrated deep learning to solve assembly positioning task such as peg-in-hole. Design/methodology/approach - Hybrid camera configuration was used to provide the global and local views. Eye-in-hand mode guided the peg to be in contact with the hole plate using 3D vision in global view. When the peg was in contact with the workpiece surface, eye-to-hand mode provided the local view to accomplish peg-hole positioning based on trained CNN. Findings - The results of assembly positioning experiments proved that the proposed method successfully distinguished the target hole from the other same size holes according to the CNN. The robot planned the motion according to the depth images and human skill guide line. The final positioning precision was good enough for the robot to carry out force controlled assembly. Practical implications - The developed framework can have an important impact on robotic assembly positioning process, which combine with the existing force-guidance assembly technology as to build a whole set of autonomous assembly technology. Originality/value - This paper proposed a new approach to the robotic assembly positioning based on 3D visual technologies and human skill integrated deep learning. Dual cameras swapping mode was used to provide visual feedback for the entire assembly motion planning process. The proposed workpiece positioning method provided an effective disturbance rejection, autonomous motion planning and increased overall performance with depth images feedback. The proposed peg-hole positioning method with human skill integrated provided the capability of target perceptual aliasing avoiding and successive motion decision for the robotic assembly manipulation.

Pomdp Based Robot Teaching for High Precision Assembly in Manufacturing Automation

Robot learning based on Partial Observable Markov Decision Process in unstructured environment

Perception of Demonstration for Automatic Programing of Robotic Assembly: Framework, Algorithm, and Validation

Robot Base Frame Calibration with a 2D Vision System for Mobile Robotic Drilling

Teaching robots to do object assembly using multi-modal 3D vision

Robot teaching playback based on machine vision and human hand demonstration

A General Mobile Manipulator Automation Framework for Flexible Manufacturing in Hostile Industrial Environments

Combining Learning from Demonstration with Learning by Exploration to Facilitate Contact-Rich Tasks

A Feature Reserved Teaching Method for Pick-Place System under Robot Operating System

Robot Programming by Demonstration with Local Human Correction for Assembly

An Online Robot Teaching Method using Static Hand Gestures and Poses

Human Skill Integrated Motion Planning of Assembly Manipulation for 6R Industrial Robot.

An efficient visual PID control approach for mobile robot

Robot Teaching by Teleoperation Based on Visual Interaction and Neural Network Learning

Online Robot Teaching with Natural Human–Robot Interaction

Online teaching system of teleoperation robot based on digital twin model

A Human-Assisted Based Visual Servo Approach For Tele-Assembly

A Method Based on Wearable Devices for Controlling Teaching of Robots for Human-robot Collaboration

Natural Teaching for Humanoid Robot Via Human-in-the-loop Scene-Motion Cross-Modal Perception.

A Robot 3C Assembly Skill Learning Method by Intuitive Human Assembly Demonstration

A Human–Robot Collaboration Method Using a Pose Estimation Network for Robot Learning of Assembly Manipulation Trajectories From Demonstration Videos