Abstract:Purpose In robot programming by demonstration (PbD) of small parts assembly tasks, the accuracy of parts poses estimated by vision-based techniques in demonstration stage is far from enough to ensure a successful execution. This paper aims to develop an inference method to improve the accuracy of poses and assembly relations between parts by integrating visual observation with computer-aided design (CAD) model. Design/methodology/approach In this paper, the authors propose a spatial information inference method called probabilistic assembly graph with optional CAD model, shorted as PAGC*, to achieve this task. Then an assembly relation extraction method from CAD model is designed, where different assembly relation descriptions in CAD model are summarized into two fundamental relations that are colinear and coplanar. The relation similarity, distance similarity and rotation similarity are adopted as the similar part matching criterions between the CAD model and the observation. The knowledge of part in CAD is used to correct that of the corresponding part in observation. The likelihood maximization estimation is used to infer the accurate poses and assembly relations based on the probabilistic assembly graph. Findings In the experiments, both simulated data and real-world data are applied to evaluate the performance of the PAGC* model. The experimental results show the superiority of PAGC* in accuracy compared with assembly graph (AG) and probabilistic assembly graph without CAD model (PAG). Originality/value The paper provides a new approach to get the accurate pose of each part in demonstration stage of the robot PbD system. By integrating information from visual observation with prior knowledge from CAD model, PAGC* ensures the success in execution stage of the PbD system.

A spatial information inference method for programming by demonstration of assembly tasks by integrating visual observation with CAD model

Probabilistic Graph Based Spatial Assembly Relation Inference for Programming of Assembly Task by Demonstration.

Perception of Demonstration for Automatic Programing of Robotic Assembly: Framework, Algorithm, and Validation

Towards Learning from Demonstration System for Parts Assembly: A Graph Based Representation for Knowledge

KGAssembly: Knowledge Graph-Driven Assembly Process Generation and Evaluation for Complex Components

Digital twin model-based smart assembly strategy design and precision evaluation for PCB kit-box build

CAD Parts-Based Assembly Modeling by Probabilistic Reasoning

Robot Programming by Demonstration with Local Human Correction for Assembly

Projection-based Augmented Reality System for Assembly Guidance and Monitoring

Multi-class Assembly Parts Recognition Using Composite Feature and Random Forest for Robot Programming by Demonstration.

Pose Estimation Algorithm of 3C Parts Based on Virtual 3D Sensor for Robot Assembly

A knowledge-based approach to assembly sequence planning

Assembly accuracy prediction based on CAD model

6D Assembly Pose Estimation by Point Cloud Registration for Robot Manipulation

Visual Guidance Method for Artificial Assembly in Visual Blind Areas Based on Augmented Reality

Combining Vision Sensing with Knowledge Database for Environmental Modeling in Dual-arm Robot Assembly Task

CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs

AR-assisted assembly method based on instance segmentation

High-Precision Pose Estimation Method of the 3C Parts by Combining 2D and 3D Vision for Robotic Grasping in Assembly Applications.

Precision Predicting and Computer Aided Adaptive Assembly

Learning Optimal Measurement and Control of Assembly Robot for Large-Scale Heavy-Weight Parts