Multiple weld seam extraction from RGB-depth images for automatic robotic welding via point cloud registration

Jiwan Kim,Jeongjin Lee,Minyoung Chung,Yeong-Gil Shin

DOI: https://doi.org/10.1007/s11042-020-10138-7

IF: 2.577

2020-11-13

Multimedia Tools and Applications

Abstract:Robotic welding technology is constantly growing with the development of vision technologies. To establish a fully automatic robotic welding system, an automated weld seam extraction process with accurate perception of the target workpiece (i.e., object) is one of the most challenging tasks. Although many methods pertaining to automatic weld seam extraction have been proposed, most of the previous studies are difficult to employ in practical systems because the algorithms are unable to simultaneously handle multiple and occluded seams from arbitrary view directions. In this study, we propose a novel method to extract weld seams based on point cloud registration from various view directions that can handle randomly occluded seams in a workpiece with multiple seam parts. We focus on the shape of the weld seams as a line or curve, which are dominant structures in the field. Initially, we extract all detectable weld seams for each image with a single view direction obtained using an RGB-depth vision sensor. Subsequently, three-dimensional points of weld seams obtained from each view direction are filtered based on the edge cues from RGB images. Finally, the extracted weld seams obtained from the various view directions are merged by employing a point cloud registration technique. The experimental results demonstrate that the proposed method outperforms a state-of-the-art method in terms of detection accuracy. Our proposed algorithm can be employed for dynamic workpiece scenes, which indicates that multiple weld seams can be successfully extracted from arbitrary view directions containing scene occlusions.

computer science, information systems, theory & methods,engineering, electrical & electronic, software engineering