A SAM-guided Two-stream Lightweight Model for Anomaly Detection

Chenghao Li,Lei Qi,Xin Geng

2024-02-29

Abstract:In industrial anomaly detection, model efficiency and mobile-friendliness become the primary concerns in real-world applications. Simultaneously, the impressive generalization capabilities of Segment Anything (SAM) have garnered broad academic attention, making it an ideal choice for localizing unseen anomalies and diverse real-world patterns. In this paper, considering these two critical factors, we propose a SAM-guided Two-stream Lightweight Model for unsupervised anomaly detection (STLM) that not only aligns with the two practical application requirements but also harnesses the robust generalization capabilities of SAM. We employ two lightweight image encoders, i.e., our two-stream lightweight module, guided by SAM's knowledge. To be specific, one stream is trained to generate discriminative and general feature representations in both normal and anomalous regions, while the other stream reconstructs the same images without anomalies, which effectively enhances the differentiation of two-stream representations when facing anomalous regions. Furthermore, we employ a shared mask decoder and a feature aggregation module to generate anomaly maps. Our experiments conducted on MVTec AD benchmark show that STLM, with about 16M parameters and achieving an inference time in 20ms, competes effectively with state-of-the-art methods in terms of performance, 98.26% on pixel-level AUC and 94.92% on PRO. We further experiment on more difficult datasets, e.g., VisA and DAGM, to demonstrate the effectiveness and generalizability of STLM.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address two key issues in industrial anomaly detection: model efficiency and mobile device friendliness. Specifically:

1. **Model Efficiency**: In practical applications, the model needs to be efficient and perform well on mobile devices.
2. **Generalization Ability of Anomaly Detection**: Existing methods struggle to handle unseen anomalies and diverse real-world patterns.

To tackle these issues, the authors propose a dual-stream lightweight model (STLM) based on Segment Anything (SAM) for unsupervised anomaly detection. This model not only meets the demands of practical applications but also leverages SAM's powerful generalization ability to effectively explore unseen anomalies and diverse normal patterns.

### Main Contributions

1. **Proposing the STLM Model**: This model not only meets the requirements of model efficiency and mobile device friendliness for practical industrial applications but also utilizes SAM's powerful generalization ability to effectively explore unseen anomalies and diverse normal patterns.
2. **Experimental Validation**: Extensive experiments on the MVTec AD, VisA, and DAGM datasets demonstrate that this method competes closely with state-of-the-art methods in terms of detection and localization, particularly excelling in parameter efficiency and inference speed, showcasing strong generalization ability.

Through these contributions, STLM becomes a promising solution for practical applications.