IDDM: An incremental dual-network detection model for in-situ inspection of large-scale complex product

Fenghua Zhang,Zhehan Chen

DOI: https://doi.org/10.1016/j.jii.2023.100463

IF: 11.718

2023-06-01

Journal of Industrial Information Integration

Abstract:Appearance inspection is crucial for quality control during the manufacturing of large complex products. In-situ visual inspection based on image processing and machine learning can significantly reduce the production costs by avoiding the stages of transshipment and relocation, etc. However, imaging and model training are challenged by the complex background, illuminance, and changing environment of the production site. In addition, large dataset is hard to be obtained with object-level annotations owing to the high cost of manual annotation in practical situations. In this paper, we proposed an Incremental Dual network Detection Model (IDDM) for efficient and high-precision inspection of the appearance of large complex product base on in-situ images. A dual network structure is used to implement the incremental training of the model based on metric learning and batch labeling of unlabeled data during the training on a small number of well-labeled samples. The regions of interest are extracted based on multi-feature and refined to improve the positional accuracy of the defects in complex background. On the public dataset, the experimental results derived on various annotation scales showed a better performance of the proposed IDDM compared to the supervised object-detection baselines. The mean Average Precision (mAP) was 51.8% with a 25% labeled ratio and the processing speed was 22 frames per second (FPS). In addition, the proposed method was applied to the defect detection of the snow groomer surface as an industrial case.

computer science, interdisciplinary applications,engineering, industrial

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve efficient and high - precision appearance defect detection with limited labeled data in on - site visual inspection during the manufacturing process of large - scale and complex products in complex backgrounds, illumination conditions and changing environments. Specifically, traditional methods rely on manual visual inspection, which is error - prone and inefficient and cannot meet the development needs of intelligent manufacturing technology. In addition, due to the high cost of obtaining large - scale datasets with object - level labels in actual situations, existing methods usually rely on large - scale datasets, which limits their applications in specific scenarios, especially when the dataset is insufficient.

To solve these problems, the author proposes an Incremental Dual - Network Detection Model (IDDM), which achieves improvements in the following ways:

1. **Dual - network structure**: By using metric learning and batch - labeling unlabeled data methods, incremental training of the model is realized during the training process, thereby reducing the dependence on large - scale datasets and achieving better performance on a small number of labeled samples.
2. **Multi - feature extraction module**: Extract the regions of interest from complex backgrounds, fuse these regions and optimize the boundaries to improve the accuracy of defect locations.

The experimental results show that the proposed IDDM model outperforms the supervised object detection baseline model on public datasets. In particular, in the case of only 25% labeling rate, the mean Average Precision (mAP) reaches 51.8% and the processing speed is 22 frames per second (FPS). In addition, this method is also applied to the actual case of surface defect detection of snow rollers.