Abstract:In modern production environments, advanced and intelligent process monitoring strategies are required to enable an unambiguous diagnosis of the process situation and thus of the final component quality. In addition, the ability to recognize the current state of product quality in real-time is an important prerequisite for autonomous and self-improving manufacturing systems. To address these needs, this study investigates a novel ensemble deep learning architecture based on convolutional neural networks (CNN), gated recurrent units (GRU) combined with high-performance classification algorithms such as k-nearest neighbors (kNN) and support vector machines (SVM). The architecture uses spatio-temporal features extracted from infrared image sequences to locate critical welding defects including lack of fusion (false friends), sagging, lack of penetration, and geometric deviations of the weld seam. In order to evaluate the proposed architecture, this study investigates a comprehensive scheme based on classical machine learning methods using manual feature extraction and state-of-the-art deep learning algorithms. Optimal hyperparameters for each algorithm are determined by an extensive grid search. Additional work is conducted to investigate the significance of various geometrical, statistical and spatio-temporal features extracted from the keyhole and weld pool regions. The proposed method is finally validated on previously unknown welding trials, achieving the highest detection rates and the most robust weld defect recognition among all classification methods investigated in this work. Ultimately, the ensemble deep neural network is implemented and optimized to operate on low-power embedded computing devices with low latency (1.1 ms), demonstrating sufficient performance for real-time applications.

Applying low rank representation based spatial pyramid matching in welding image classification

Fast Low Rank Representation Based Spatial Pyramid Matching for Image Classification.

A Feature Matching Algorithm Applied to Welding Trajectory Revising

Weld seam object detection system based on the fusion of 2D images and 3D point clouds using interpretable neural networks

Image Classification Method Rationally Utilizing Spatial Information of the Image

Weld Seam Detection Method with Rotational Region Proposal Network.

High-resolution Weld Defect Detection with RSU-MLP and Dynamic Kernel Supervision

Patent Image Classification Using Local-Constrained Linear Coding and Spatial Pyramid Matching

Welding Surface Inspection of Armatures via CNN and Image Comparison

Online detection of weld surface defects based on improved incremental learning approach

Performance analysis of object detection algorithms for robotic welding applications in planar environment

Enhancing Weld Inspection Through Comparative Analysis of Traditional Algorithms and Deep Learning Approaches

Weld Image Recognition Based on Deep Learning

A weld feature points detection method based on improved YOLO for welding robots in strong noise environment

A Spatio-Temporal Ensemble Deep Learning Architecture for Real-Time Defect Detection during Laser Welding on Low Power Embedded Computing Boards

Discerning Weld Seam Profiles from Strong Arc Background for the Robotic Automated Welding Process via Visual Attention Features

Semantic-Spatial Matching for Image Classification

3D reconstruction method of welding area by fusion of coding raster and semantic segmentation network

A Weld Joint Type Identification Method for Visual Sensor Based on Image Features and SVM

Research on a Feature Point Detection Algorithm for Weld Images Based on Deep Learning

Prediction of high power laser welding status based on PCA and SVM classification of multiple sensors