Long-Tailed Anomaly Detection with Learnable Class Names

Chih-Hui Ho,Kuan-Chuan Peng,Nuno Vasconcelos

2024-03-29

Abstract:Anomaly detection (AD) aims to identify defective images and localize their defects (if any). Ideally, AD models should be able to detect defects over many image classes; without relying on hard-coded class names that can be uninformative or inconsistent across datasets; learn without anomaly supervision; and be robust to the long-tailed distributions of real-world applications. To address these challenges, we formulate the problem of long-tailed AD by introducing several datasets with different levels of class imbalance and metrics for performance evaluation. We then propose a novel method, LTAD, to detect defects from multiple and long-tailed classes, without relying on dataset class names. LTAD combines AD by reconstruction and semantic AD modules. AD by reconstruction is implemented with a transformer-based reconstruction module. Semantic AD is implemented with a binary classifier, which relies on learned pseudo class names and a pretrained foundation model. These modules are learned over two phases. Phase 1 learns the pseudo-class names and a variational autoencoder (VAE) for feature synthesis that augments the training data to combat long-tails. Phase 2 then learns the parameters of the reconstruction and classification modules of LTAD. Extensive experiments using the proposed long-tailed datasets show that LTAD substantially outperforms the state-of-the-art methods for most forms of dataset imbalance. The long-tailed dataset split is available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Main Problems Addressed by the Paper

This paper proposes a new method to address the long-tail distribution problem in the field of Anomaly Detection (AD). Specifically, the paper addresses the following key issues:

1. **Multi-class Anomaly Detection**: Existing methods typically require training a separate model for each image category to detect anomalies. The proposed method (LTAD) aims to handle multiple categories of anomaly detection tasks using a single model.

2. **Independence from Hard-coded Category Names**: Many existing methods rely on predefined textual prompts to distinguish between normal and abnormal situations, but these prompts may be inconsistent or insufficient across different datasets. LTAD overcomes this limitation by learning pseudo-category names.

3. **No Need for Anomaly Supervision**: The proposed method can be trained without any anomaly samples.

4. **Handling Long-tail Distribution in Real-world Applications**: Real-world data often exhibit a long-tail distribution, where the number of samples in some categories far exceeds that in others. This imbalance can lead to poor model performance on minority categories. The paper introduces the long-tail anomaly detection problem and designs corresponding evaluation datasets and performance metrics to address this challenge.

### Main Contributions

1. **Design of a New Method**: The LTAD method is proposed, combining reconstruction-based anomaly detection and semantic anomaly detection techniques. It can handle multi-class anomaly detection tasks and solve the problem of unclear or missing category names by learning pseudo-category names.

2. **Data Augmentation Strategy**: To address the data scarcity problem caused by long-tail distribution, the paper proposes a novel data augmentation strategy, including using Variational Autoencoders (VAE) to synthesize features and learning pseudo-category names to improve the model's sensitivity to long-tail categories.

3. **Empirical Studies**: Extensive experiments validate the superior performance of LTAD under various imbalanced configurations, especially showing significant advantages over existing methods in long-tail settings.

In summary, the main goal of this paper is to address the challenge of long-tail distribution in multi-class anomaly detection and propose a comprehensive solution for it.