Abstract:In the field of medical decision-making, precise anomaly detection in medical imaging plays a pivotal role in aiding clinicians. However, previous work is reliant on large-scale datasets for training anomaly detection models, which increases the development cost. This paper first focuses on the task of medical image anomaly detection in the few-shot setting, which is critically significant for the medical field where data collection and annotation are both very expensive. We propose an innovative approach, MediCLIP, which adapts the CLIP model to few-shot medical image anomaly detection through self-supervised fine-tuning. Although CLIP, as a vision-language model, demonstrates outstanding zero-/fewshot performance on various downstream tasks, it still falls short in the anomaly detection of medical images. To address this, we design a series of medical image anomaly synthesis tasks to simulate common disease patterns in medical imaging, transferring the powerful generalization capabilities of CLIP to the task of medical image anomaly detection. When only few-shot normal medical images are provided, MediCLIP achieves state-of-the-art performance in anomaly detection and location compared to other methods. Extensive experiments on three distinct medical anomaly detection tasks have demonstrated the superiority of our approach. The code is available at

What problem does this paper attempt to address?

The paper attempts to address the problem of how to achieve efficient and accurate anomaly detection and localization in medical images with few-shot samples. Traditional methods usually require a large amount of annotated data to train models, which is both expensive and time-consuming in the medical field. MediCLIP achieves effective detection and localization of anomalies in medical images with only a small number of normal images by self-supervised fine-tuning of the CLIP model and using synthetic anomaly images to simulate common disease patterns. Specifically, the main contributions of the paper include: 1. **Proposing the MediCLIP model**: This is an innovative approach that adapts the CLIP model to the few-shot medical image anomaly detection task, improving the model's generalization ability through self-supervised fine-tuning. 2. **Designing a multi-task anomaly synthesis strategy**: By generating simulated anomaly images through various synthesis tasks, including CutPaste, GaussIntensityChange, and Source, it effectively simulates common anomaly patterns in medical images. 3. **Introducing learnable prompts and adapters**: Using learnable prompts and adapters to enhance the model's performance on medical images, avoiding complex manual prompt design, and enhancing the model's multi-scale lesion localization capability. 4. **Proving effectiveness through experiments**: Extensive experiments were conducted on three different medical image datasets (CheXpert, BrainMRI, and BUSI), and the results show that MediCLIP significantly outperforms existing methods in a few-shot setting, especially in anomaly detection and localization. In summary, the paper aims to reduce the development cost of medical image anomaly detection and improve model performance through innovative technical means, thereby providing strong support for clinical decision-making.

MediCLIP: Adapting CLIP for Few-shot Medical Image Anomaly Detection

Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images

MedicalCLIP: Anomaly-Detection Domain Generalization with Asymmetric Constraints

AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection

SimCLIP: Refining Image-Text Alignment with Simple Prompts for Zero-/Few-shot Anomaly Detection

Exploring Zero-Shot Anomaly Detection with CLIP in Medical Imaging: Are We There Yet?

CLIP3D-AD: Extending CLIP for 3D Few-Shot Anomaly Detection with Multi-View Images Generation

WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation

Position-Guided Prompt Learning for Anomaly Detection in Chest X-Rays

AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection

Anomaly Detection by Adapting a pre-trained Vision Language Model

Dual-Image Enhanced CLIP for Zero-Shot Anomaly Detection

CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection

CLIP-FSAC++: Few-Shot Anomaly Classification with Anomaly Descriptor Based on CLIP

CLIP-FSAC: Boosting CLIP for Few-Shot Anomaly Classification with Synthetic Anomalies

Contrastive Language Prompting to Ease False Positives in Medical Anomaly Detection

MedCLIP: Contrastive Learning from Unpaired Medical Images and Text

DiffCLIP: Few-shot Language-driven Multimodal Classifier

CLIP in Medical Imaging: A Comprehensive Survey

FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language Model

VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection