Cascaded foreign object detection in manufacturing processes using convolutional neural networks and synthetic data generation methodology

Jixiang Tang,Huan Zhou,Tiankui Wang,Zhenxun Jin,Youli Wang,Xuanyin Wang
DOI: https://doi.org/10.1007/s10845-022-01976-3
IF: 8.3
2022-06-24
Journal of Intelligent Manufacturing
Abstract:Foreign object detection in manufacturing processes based on machine vision remains a challenge. The vastly different foreign objects and the complex background, as well as the scarcity of images with foreign objects constrain the application of traditional and deep learning methods, respectively. This paper discusses a novel method for intelligent foreign object detection and automatic data generation. A cascaded convolutional neural network to detect foreign objects on the surface of the tobacco pack is proposed. The cascaded network transforms the inspection into a two-stage YOLO based object detection, consisting of the tobacco pack localization and the foreign object detection. To address the scarcity of images with foreign objects, several data augmentation methods are introduced to avoid overfitting. Furthermore, a data generation methodology based on homography transformation and image fusion is developed to generate synthetic images with foreign objects. Models trained using synthetic images generated by this method show superior performance, which presents a viable approach to detecting newly introduced foreign objects. Extensive experimental results and comparisons on the tobacco pack foreign object dataset with several state-of-the-art methods demonstrate the effectiveness and superiority of the proposed method. The proposed cascaded foreign object detection network and synthetic data generation methodology have the potential for widespread applications.
engineering, manufacturing,computer science, artificial intelligence
What problem does this paper attempt to address?