Abstract:In recent years, unsupervised anomaly detection based on knowledge distillation has gained special attention and some promising results have been reported in the literature. However, there is still room to improve the sensitivity of the model to anomalies. To do so, in this paper, a novel two-stage training method in terms of reverse knowledge distillation is proposed for anomaly detection and localization. Firstly, self-supervised mask training is introduced after the initial training of reverse knowledge distillation, which contributes greatly to the model detection against random unknown anomalies by self-simulating anomalies and forcing repair so as to reinforce learning single-category prototype patterns. Then, with the aim to facilitate the anomaly localization, an anomaly feature diffusion module is employed, which strengthens the correlation between pixels and helps spread the anomaly information to the surrounding area by covering the central pixel and reconstructing the representation for features after diffused. Furthermore, inspired by the human memory mechanism, an innovative normalized embedding memory bank is adopted to regulate the low-dimensional representations after embedding the encoding, inhibit the flow of anomalous information to the student decoder, and encourage the high-quality reconstruction of the model. Finally, the contextual similarity loss is used to guide the student model to learn knowledge representations from a contextual perspective, capture higher-order similarities between teachers and students, and delicately evaluate the differences between teachers and students. The empirical experiments conducted on the MVTec dataset show that the proposed SSMRKD method can achieve the best performance compared to other state-of-the-art methods, meanwhile extensive experiments of the ablation study validate the contribution of each component of the model. In addition, the advanced performance achieved on four commonly used datasets verifies the generalizability of the model in the industrial domain. Overall, the proposed SSMRKD method has significant advantages over the state-of-the-art anomaly detection methods.

Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection

Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detection

Pull & Push: Leveraging Differential Knowledge Distillation for Efficient Unsupervised Anomaly Detection and Localization

Unsupervised anomaly detection and localization via bidirectional knowledge distillation

Abnormal-Aware Loss and Full Distillation for Unsupervised Anomaly Detection Based on Knowledge Distillation

Remembering Normality: Memory-guided Knowledge Distillation for Unsupervised Anomaly Detection

UNSUPERVISED ANOMALY DETECTION WITH SELF-TRAINING AND KNOWLEDGE DISTILLATION

Large Language Models for Anomaly Detection in Computational Workflows: from Supervised Fine-Tuning to In-Context Learning

Large language models can be zero-shot anomaly detectors for time series?

Two-stage reverse knowledge distillation incorporated and Self-Supervised Masking strategy for industrial anomaly detection

Anomaly Detection of Tabular Data Using LLMs

Label-Efficient Interactive Time-Series Anomaly Detection

Self-Supervised Time-Series Anomaly Detection Using Learnable Data Augmentation

Multi-Scale Feature Distillation for Anomaly Detection

Autoencoder-Like Knowledge Distillation Network for Anomaly Detection

Advancing Pre-trained Teacher: Towards Robust Feature Discrepancy for Anomaly Detection

Taming Pre-trained LLMs for Generalised Time Series Forecasting via Cross-modal Knowledge Distillation

Anomaly detection based on multi-teacher knowledge distillation

Anomaly Detection via Reverse Distillation from One-Class Embedding

Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection

LGAT: A Novel Model for Multivariate Time Series Anomaly Detection with Imporved Anomaly Transformer and Learning Graph Structures