Abstract:The key to OOD detection has two aspects: generalized feature representation and precise category description. Recently, vision-language models such as CLIP provide significant advances in both two issues, but constructing precise category descriptions is still in its infancy due to the absence of unseen categories. This work introduces two hierarchical contexts, namely perceptual context and spurious context, to carefully describe the precise category boundary through automatic prompt tuning. Specifically, perceptual contexts perceive the inter-category difference (e.g., cats vs apples) for current classification tasks, while spurious contexts further identify spurious (similar but exactly not) OOD samples for every single category (e.g., cats vs panthers, apples vs peaches). The two contexts hierarchically construct the precise description for a certain category, which is, first roughly classifying a sample to the predicted category and then delicately identifying whether it is truly an ID sample or actually OOD. Moreover, the precise descriptions for those categories within the vision-language framework present a novel application: CATegory-EXtensible OOD detection (CATEX). One can efficiently extend the set of recognizable categories by simply merging the hierarchical contexts learned under different sub-task settings. And extensive experiments are conducted to demonstrate CATEX's effectiveness, robustness, and category-extensibility. For instance, CATEX consistently surpasses the rivals by a large margin with several protocols on the challenging ImageNet-1K dataset. In addition, we offer new insights on how to efficiently scale up the prompt engineering in vision-language models to recognize thousands of object categories, as well as how to incorporate large language models (like GPT-3) to boost zero-shot applications. Code will be made public soon.

Matching Words for Out-of-distribution Detection

Out-of-Distribution Detection Using Peer-Class Generated by Large Language Model

MOODv2: Masked Image Modeling for Out-of-Distribution Detection

Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP

Exploiting Diffusion Prior for Out-of-Distribution Detection

CLIP-driven Outliers Synthesis for few-shot OOD detection

Detecting Out-of-Distribution Examples Via Class-Conditional Impressions Reappearing

Category-Extensible Out-of-Distribution Detection via Hierarchical Context Descriptions

Class Relevance Learning For Out-of-distribution Detection

A Simple Test-Time Method for Out-of-Distribution Detection

Learning by Erasing: Conditional Entropy based Transferable Out-Of-Distribution Detection

Out-of-Distribution Detection with Deep Nearest Neighbors

Exploring Large Language Models for Multi-Modal Out-of-Distribution Detection

Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution

Out-of-Distribution Detection by Principal Component Correspondence

Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection

Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Models

A Unified Approach to Semi-Supervised Out-of-Distribution Detection

General-Purpose Multi-Modal OOD Detection Framework

CASE: Exploiting Intra-class Compactness and Inter-class Separability of Feature Embeddings for Out-of-Distribution Detection

Continual Unsupervised Out-of-Distribution Detection