Non-Linear Outlier Synthesis for Out-of-Distribution Detection

Lars Doorenbos,Raphael Sznitman,Pablo Márquez-Neila

2024-11-20

Abstract:The reliability of supervised classifiers is severely hampered by their limitations in dealing with unexpected inputs, leading to great interest in out-of-distribution (OOD) detection. Recently, OOD detectors trained on synthetic outliers, especially those generated by large diffusion models, have shown promising results in defining robust OOD decision boundaries. Building on this progress, we present NCIS, which enhances the quality of synthetic outliers by operating directly in the diffusion's model embedding space rather than combining disjoint models as in previous work and by modeling class-conditional manifolds with a conditional volume-preserving network for more expressive characterization of the training distribution. We demonstrate that these improvements yield new state-of-the-art OOD detection results on standard ImageNet100 and CIFAR100 benchmarks and provide insights into the importance of data pre-processing and other key design choices. We make our code available at \url{<a class="link-external link-https" href="https://github.com/LarsDoorenbos/NCIS" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively detect out - of - distribution (OOD) samples in deep - learning classifiers. Specifically, when a trained classifier encounters new samples that do not belong to the distribution of the training data, it usually wrongly classifies them into known categories, which leads to the unreliability of the model. Therefore, the goal of the paper is to improve the classifier's ability to recognize OOD samples, so as to enhance the reliability and robustness of the system.

To achieve this goal, the authors propose a new method - Non - Linear Class - wise Invariant Sampling (NCIS). NCIS improves existing OOD detection techniques by generating high - quality synthetic outliers in the embedding space of the diffusion model. The specific contributions are as follows:

1. **Diffusion Model Embedding**: The authors propose a method of directly using the diffusion model to generate image embeddings, avoiding the dependence on external embedding functions or surrogate models. This method can more accurately represent the distribution area of the training data in the conditional space of the diffusion model.

2. **Non - Linear Parameter Distribution**: Inspired by the use of non - linear invariants for OOD detection, the authors introduce a Conditional Volume - Preserving Network (cVPN) to fit the class - conditional manifold and show how to use it to fit any complex class - conditional distribution of training image embeddings. This complex distribution makes it possible to sample more accurately along the ID/OOD boundary in the conditional space.

3. **Generating OOD Samples**: By combining the above two techniques, NCIS can generate realistic and diverse outliers, thereby enhancing the classifier's ability to detect OOD samples.

4. **Experimental Verification**: The paper conducts experiments on two benchmark datasets (ImageNet - 100 and CIFAR - 100), and the results show that NCIS has reached a new state - of - the - art level in the OOD detection task. In addition, the ablation study confirms the individual influence of each component, further verifying the effectiveness of the method.

In conclusion, through innovative technical means, this paper significantly improves the classifier's performance in detecting OOD samples and provides a new solution for improving the reliability and robustness of deep - learning models.