Abstract:Recent studies of multimodal industrial anomaly detection (IAD) based on 3D point clouds and RGB images have highlighted the importance of exploiting the redundancy and complementarity among modalities for accurate classification and segmentation. However, achieving multimodal IAD in practical production lines remains a work in progress. It is essential to consider the trade-offs between the costs and benefits associated with the introduction of new modalities while ensuring compatibility with current processes. Existing quality control processes combine rapid in-line inspections, such as optical and infrared imaging with high-resolution but time-consuming near-line characterization techniques, including industrial CT and electron microscopy to manually or semi-automatically locate and analyze defects in the production of Li-ion batteries and composite materials. Given the cost and time limitations, only a subset of the samples can be inspected by all in-line and near-line methods, and the remaining samples are only evaluated through one or two forms of in-line inspection. To fully exploit data for deep learning-driven automatic defect detection, the models must have the ability to leverage multimodal training and handle incomplete modalities during inference. In this paper, we propose CMDIAD, a Cross-Modal Distillation framework for IAD to demonstrate the feasibility of a Multi-modal Training, Few-modal Inference (MTFI) pipeline. Our findings show that the MTFI pipeline can more effectively utilize incomplete multimodal information compared to applying only a single modality for training and inference. Moreover, we investigate the reasons behind the asymmetric performance improvement using point clouds or RGB images as the main modality of inference. This provides a foundation for our future multimodal dataset construction with additional modalities from manufacturing scenarios.

Multi-modal background-aware for defect semantic segmentation with limited data

Multivariate image analysis in gaussian multi-scale space for defect detection

A Sub-Region Unet for Weak Defects Segmentation with Global Information and Mask-Aware Loss.

Learn to Differ: Sim2Real Small Defection Segmentation Network

Self-supervised assisted multi-task learning network for one-shot defect segmentation with fake defect generation

Few-Shot Defect Segmentation Leveraging Abundant Normal Training Samples Through Normal Background Regularization and Crop-and-Paste Operation

Multi-scale Attention and Dilation Network for Small Defect Detection

Small-scale defect detection in industrial environment based on lightweight deep learning network

Interactive Defect Segmentation in X-Ray Images Based on Deep Learning

AnomalySeg: Deep Learning-Based Fast Anomaly Segmentation Approach for Surface Defect Detection

Efficient and accurate semi-supervised semantic segmentation for industrial surface defects

LERENet: Eliminating Intra-class Differences for Metal Surface Defect Few-shot Semantic Segmentation

Data-Driven Semantic Segmentation Method for Detecting Metal Surface Defects

DDSNet: Deep Dual-Branch Networks for Surface Defect Segmentation

Multi-surface defect detection for universal joint bearings via multimodal feature and deep transfer learning

Surface Defect Detection and Semantic Segmentation with a Novel Lightweight Deep Neural Network

An Adaptive Image Segmentation Network for Surface Defect Detection

Incomplete Multimodal Industrial Anomaly Detection via Cross-Modal Distillation

Change-Aware Siamese Network for Surface Defects Segmentation under Complex Background

Yolo-sd: simulated feature fusion for few-shot industrial defect detection based on YOLOv8 and stable diffusion

An Efficient End-to-End Multitask Network Architecture for Defect Inspection