Joint Weakly and Fully Supervised Learning for Surface Defect Segmentation from Images

Bin Hu,Xinggang Wang,Wenyong Yu
DOI: https://doi.org/10.1016/j.image.2022.116807
2022-01-01
Abstract:Surface defect segmentation from industrial images based on deep learning has been rapidly developed in recent years. However, the related methods depend heavily on a large number of costly pixel-level labels. This paper introduces a method that uses abundant weakly supervised images and a few fully supervised images to reduce the labeling expense and obtain an excellent segmentation result. First, we propose a novel weakly supervised semantic segmentation method, the multistage squeeze-and-excitation (SE)-augmented adaptive Lp norm class activation map (multistage SALN-CAM), which adopts both position-level and channel-level attention mechanisms. Based on multistage SALN-CAM, a Siamese network is designed to jointly train fully supervised and weakly supervised images simultaneously with the help of an auxiliary cross-field and cross -attention (CFCA) network that maps features from the classification field to the segmentation field with a cross-attention mechanism. Finally, we retrain a fully supervised segmentation model using images with pixel -level labels or pseudo-pixel-level labels generated by the Siamese network. In our experiments, based on the Severstal: Steel Defect Detection dataset, our method, in which 25% of the images (2512) have pixel-level labels and 75% (7544) of the images have class-level labels, obtains an mDice of 94.01% on the test set. This result completely surpasses that of the fully supervised method, which uses 4775 images; thus, our method reduces the labeling expense by 37%.
What problem does this paper attempt to address?