Abstract:At present, sensor recognition technology has become a research hotspot in welding process monitoring. However, most of the existing defect recognition methods are based on single-sensor information, which hinders the further improvement of recognition accuracy and weakens the anti-interference ability under complex conditions. To overcome the shortcomings of single-sensor recognition technology, multi-source information fusion technology can exploit the advantages of different types of sensors to achieve more comprehensive and accurate monitoring of the welding process. However, given the process of fusion of different information, the feature mining mechanism of the deep learning model is still under-explained, and the direct complementarity of the different information remains unclear. In this paper, we propose a hybrid model that uses the fusion of molten pool image and voltage signals to quickly and accurately identify three types of defects in lap MIG welding: burn -through, weld bias, and uneven width. Firstly, a synchronized trigger device is used to acquire molten pool images and weld voltage information, and the voltage data is sliced and matched with the molten pool image data. Secondly, the Short-Time Fourier Transform (STFT) is applied to explore the frequency variation components in the voltage data, and the resulting time-frequency matrix is transformed into a time spectrogram. Finally, the molten pool image and the time spectrogram are simultaneously fed into the hybrid CNN model for online monitoring of welding defect states. The test results show that the average accuracy of the hybrid CNN model in detecting five different welding states reaches 98.87 %, which is significantly higher than three other models based on single sensor information. The visualization of the confusion matrix and Grad-CAM shows that the hybrid CNN model based on multi-source heterogeneous information has a higher accuracy for detecting the transition states that are difficult to detect in welding, and has a more reasonable mechanism for allocating attention, so it can achieve a more accurate and balanced recognition effect in the whole welding manufacturing process. These studies can provide guidance for quality monitoring of welding processes based on multi-source sensor technology, which can be extended to the field of arc additive monitoring.

AF-FTTSnet: An end-to-end two-stream convolutional neural network for online quality monitoring of robotic welding

A Novel Quality Monitoring Approach Based on Multigranularity Spatiotemporal Attentive Representation Learning During Climbing GTAW

Prediction of Welding Quality Characteristics During Pulsed GTAW Process of Aluminum Alloy by Multisensory Fusion and Hybrid Network Model

Early Event Detection in a Deep-Learning Driven Quality Prediction Model for Ultrasonic Welding

Intelligent Seam Tracking in Foils Joining Based on Spatial–temporal Deep Learning from Molten Pool Serial Images

Online Visual Monitoring Method for Liquid Rocket Engine Nozzle Welding Based on a Multi-Task Deep Learning Model

On-line Defect Recognition of MIG Lap Welding for Stainless Steel Sheet Based on Weld Image and CMT Voltage: Feature Fusion and Attention Weights Visualization

DSNet: A dynamic squeeze network for real-time weld seam image segmentation

Welding Defect Monitoring Based on Multi-Scale Feature Fusion of Molten Pool Videos

Online monitoring system for welding states of bottom-locking joints in high-speed trains via multi-information fusion and 3DCNN

Real-time recognition of arc weld pool using image segmentation network

A Spatio-Temporal Ensemble Deep Learning Architecture for Real-Time Defect Detection during Laser Welding on Low Power Embedded Computing Boards

Defect monitoring of high-power laser-arc hybrid welding process based on an improved channel attention convolutional neural network

A weld feature points detection method based on improved YOLO for welding robots in strong noise environment

STMA-Net: A Spatial Transformation-Based Multiscale Attention Network for Complex Defect Detection With X-Ray Images

Real-Time Quality Monitoring of Ultrathin Sheets Edge Welding Based on Microvision Sensing and SOCIFS-SVM.

WeldNet: a lightweight deep learning model for welding defect recognition

TIG Stainless Steel Molten Pool Contour Detection and Weld Width Prediction Based on Res-Seg

Real-Time Recognition of Molten Pools Based on Improved DeepLabV3+ in Keyhole Tungsten Inert Gas Welding Applications

Deep Learning-Based Defects Detection in Keyhole TIG Welding with Enhanced Vision

Weld Penetration State Identification Based on Time Series Multi-Source Data Fusion