Abstract:Purpose In robot programming by demonstration (PbD) of small parts assembly tasks, the accuracy of parts poses estimated by vision-based techniques in demonstration stage is far from enough to ensure a successful execution. This paper aims to develop an inference method to improve the accuracy of poses and assembly relations between parts by integrating visual observation with computer-aided design (CAD) model. Design/methodology/approach In this paper, the authors propose a spatial information inference method called probabilistic assembly graph with optional CAD model, shorted as PAGC*, to achieve this task. Then an assembly relation extraction method from CAD model is designed, where different assembly relation descriptions in CAD model are summarized into two fundamental relations that are colinear and coplanar. The relation similarity, distance similarity and rotation similarity are adopted as the similar part matching criterions between the CAD model and the observation. The knowledge of part in CAD is used to correct that of the corresponding part in observation. The likelihood maximization estimation is used to infer the accurate poses and assembly relations based on the probabilistic assembly graph. Findings In the experiments, both simulated data and real-world data are applied to evaluate the performance of the PAGC* model. The experimental results show the superiority of PAGC* in accuracy compared with assembly graph (AG) and probabilistic assembly graph without CAD model (PAG). Originality/value The paper provides a new approach to get the accurate pose of each part in demonstration stage of the robot PbD system. By integrating information from visual observation with prior knowledge from CAD model, PAGC* ensures the success in execution stage of the PbD system.

Visual Guidance Method for Artificial Assembly in Visual Blind Areas Based on Augmented Reality

AR-Assisted Guidance for Assembly and Maintenance of Avionics Equipment

A spatial information inference method for programming by demonstration of assembly tasks by integrating visual observation with CAD model

Efficient evaluation of misalignment between real and virtual objects for HMD-Based AR assembly assistance system

AR-assisted assembly method based on instance segmentation

Perception of Demonstration for Automatic Programing of Robotic Assembly: Framework, Algorithm, and Validation

Development and Application of Parts Assembly Guidance System Based on Augmented Reality

A Natural Bare-Hand Interaction Method With Augmented Reality for Constraint-Based Virtual Assembly

A Digital Twin-Driven Guidance Method for Human-Machine Collaborative Assembly Operations Based on Machine Learning and Computer Vision

Interactive task planning in virtual assembly.

Deep learning-based augmented reality work instruction assistance system for complex manual assembly

A novel adaptive visualization method based on user intention in AR manual assembly

MIVAS: A Multi-Modal Immersive Virtual Assembly System

Probabilistic Graph Based Spatial Assembly Relation Inference for Programming of Assembly Task by Demonstration.

Real-time occlusion handling for augmented reality assistance assembly systems with monocular images

A natural bare-hand interface-enabled interactive AR assembly guidance

Equipment Assembly Recognition for Augmented Reality Guidance

Research and Implementation on Key Technologies of Assembly Assistance System based on Augmented Reality

A posture detection method for augmented reality–aided assembly based on YOLO-6D

Active Assembly Guidance with Online Video Parsing

Integrated Registration and Occlusion Handling Based on Deep Learning for Augmented Reality Assisted Assembly Instruction