Abstract:Due to the data imbalance and the diversity of defects, student-teacher networks (S-T) are favored in unsupervised anomaly detection, which explores the discrepancy in feature representation derived from the knowledge distillation process to recognize anomalies. However, vanilla S-T network is not stable. Employing identical structures to construct the S-T network may weaken the representative discrepancy on anomalies. But using different structures can increase the likelihood of divergent performance on normal data. To address this problem, we propose a novel dual-student knowledge distillation (DSKD) architecture. Different from other S-T networks, we use two student networks a single pre-trained teacher network, where the students have the same scale but inverted structures. This framework can enhance the distillation effect to improve the consistency in recognition of normal data, and simultaneously introduce diversity for anomaly representation. To explore high-dimensional semantic information to capture anomaly clues, we employ two strategies. First, a pyramid matching mode is used to perform knowledge distillation on multi-scale feature maps in the intermediate layers of networks. Second, an interaction is facilitated between the two student networks through a deep feature embedding module, which is inspired by real-world group discussions. In terms of classification, we obtain pixel-wise anomaly segmentation maps by measuring the discrepancy between the output feature maps of the teacher and student networks, from which an anomaly score is computed for sample-wise determination. We evaluate DSKD on three benchmark datasets and probe the effects of internal modules through ablation experiments. The results demonstrate that DSKD can achieve exceptional performance on small models like ResNet18 and effectively improve vanilla S-T networks.

Uninformed Students: Student-Teacher Anomaly Detection With Discriminative Latent Embeddings

Pull & Push: Leveraging Differential Knowledge Distillation for Efficient Unsupervised Anomaly Detection and Localization

Anomaly Detection in 3D Point Clouds using Deep Geometric Descriptors

Learning Discrimination from Contaminated Data: Multi-Instance Learning for Unsupervised Anomaly Detection

Reconstructed Student-Teacher and Discriminative Networks for Anomaly Detection

VDKD: A ViT-Based Student-Teacher Knowledge Distillation for Multi-Texture Class Anomaly Detection

DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection.

Memoryless Multimodal Anomaly Detection via Student–Teacher Network and Signed Distance Learning

Reconstruction Student with Attention for Student-Teacher Pyramid Matching

Dual-Student Knowledge Distillation Networks for Unsupervised Anomaly Detection

Structural Teacher-Student Normality Learning for Multi-Class Anomaly Detection and Localization

Teacher-Student Network for 3D Point Cloud Anomaly Detection with Few Normal Samples

A Diffusion-Based Framework for Multi-Class Anomaly Detection

Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection

Self-Supervised Anomaly Detection from Anomalous Training Data via Iterative Latent Token Masking

GANomaly: Semi-supervised Anomaly Detection via Adversarial Training

Unsupervised anomaly detection and localization via bidirectional knowledge distillation

Dual-student knowledge distillation for visual anomaly detection

Advancing Pre-trained Teacher: Towards Robust Feature Discrepancy for Anomaly Detection

The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection

Attention-Guided Perturbation for Unsupervised Image Anomaly Detection