Zero-Shot Anomaly Detection with Pre-trained Segmentation Models

Matthew Baugh,James Batten,Johanna P. Müller,Bernhard Kainz

2023-06-16

Abstract:This technical report outlines our submission to the zero-shot track of the Visual Anomaly and Novelty Detection (VAND) 2023 Challenge. Building on the performance of the WINCLIP framework, we aim to enhance the system's localization capabilities by integrating zero-shot segmentation models. In addition, we perform foreground instance segmentation which enables the model to focus on the relevant parts of the image, thus allowing the models to better identify small or subtle deviations. Our pipeline requires no external data or information, allowing for it to be directly applied to new datasets. Our team (Variance Vigilance Vanguard) ranked third in the zero-shot track of the VAND challenge, and achieve an average F1-max score of 81.5/24.2 at a sample/pixel level on the VisA dataset.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The main goal of this paper is to improve zero-shot anomaly detection methods, particularly the ability to localize anomalies in images. Specifically, the authors extend the existing WinCLIP framework by integrating a zero-shot segmentation model to enhance the accuracy of anomaly localization. This approach is especially suitable for industrial anomaly detection scenarios, where only a small number of normal samples are often available.

Below is an overview of the key issues addressed by this research:
1. **Improve localization accuracy**: By combining zero-shot segmentation techniques, this method can more precisely locate anomalous regions in images.
2. **Focus on relevant parts of the image**: Through foreground instance segmentation, the model can focus on parts of the image related to anomaly detection, which helps in better identifying smaller or subtle deviations.
3. **Enhance model generalization**: The proposed method does not require additional data or information, meaning it can be easily applied to new datasets.
4. **Improve performance under small sample conditions**: For situations where training is based on only a small number of normal samples, this method aims to enhance the model's performance to overcome limitations in real-world applications.

In summary, this research aims to achieve more precise anomaly localization in zero-shot anomaly detection tasks by integrating zero-shot segmentation models and improving the WinCLIP framework, resulting in significant performance improvements on the VisA dataset.