Beyond Perceptual Distances: Rethinking Disparity Assessment for Out-of-Distribution Detection with Diffusion Models

Kun Fang,Qinghua Tao,Zuopeng Yang,Xiaolin Huang,Jie Yang

2024-11-19

Abstract:Out-of-Distribution (OoD) detection aims to justify whether a given sample is from the training distribution of the classifier-under-protection, i.e., In-Distribution (InD), or from OoD. Diffusion Models (DMs) are recently utilized in OoD detection by using the perceptual distances between the given image and its DM generation. DM-based methods bring fresh insights to the field, yet remain under-explored. In this work, we point out two main limitations in DM-based OoD detection methods: (i) the perceptual metrics on the disparities between the given sample and its generation are devised only at human-perceived levels, ignoring the abstract or high-level patterns that help better reflect the intrinsic disparities in distribution; (ii) only the raw image contents are taken to measure the disparities, while other representations, i.e., the features and probabilities from the classifier-under-protection, are easy to access at hand but are ignored. To this end, our proposed detection framework goes beyond the perceptual distances and looks into the deep representations from the classifier-under-protection with our novel metrics devised correspondingly, leading to more informative disparity assessments between InD and OoD. An anomaly-removal strategy is integrated to remove the abnormal OoD information in the generation, further enhancing the distinctiveness of disparities. Our work has demonstrated state-of-the-art detection performances among DM-based methods in extensive experiments.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to use Diffusion Models (DMs) for more effective Out - of - Distribution (OoD) detection. Specifically, the author points out two main limitations of the existing DM - based OoD detection methods and proposes a new framework to overcome these problems.

### 1. Limitations of Existing Methods

1. **Limitations of Perceptual Distance**:
- The existing DM - based OoD detection methods only evaluate the differences between the generated image and the input image at the human - perceptual level, ignoring abstract or high - level patterns that can better reflect the intrinsic differences between distributions.

2. **Limitations of Representation Forms**:
- These methods only use the original image content to measure the differences, ignoring other easily accessible representation forms such as features and probabilities under the protection of classifiers.

### 2. The Proposed New Framework

To solve the above problems, the author proposes a new framework named **Diffusion Distribution Disparity (D3)**. The main improvements of this framework include:

1. **Beyond Perceptual Distance**:
- It not only considers the original image content but also uses the deep representations (feature space and probability space) under the protection of classifiers to evaluate the differences.

2. **Novel Metric Methods**:
- In the feature space, the ℓ2 distance is used to measure the feature differences.
- In the probability space, the Kullback - Leibler (KL) divergence is used to measure the differences in probability distributions.

3. **Anomaly Removal Strategy**:
- An anomaly removal strategy is integrated to remove the abnormal OOD information in the generated image, thereby enhancing the significance of the distribution differences.

### 3. Mathematical Formula Representation

- **ℓ2 Distance in Feature Space**:
\[
\epsilon_{\ell_2}(x, \hat{x})=\left\|\frac{\hat{h}_x}{\|\hat{h}_x\|_2}-\frac{h_x}{\|h_x\|_2}\right\|_2
\]

- **KL Divergence in Probability Space**:
\[
\epsilon_{KL}(x, \hat{x})=\frac{D_{KL}(\hat{g}_x \| u)}{D_{KL}(g_x \| u)}
\]
where \(u\) is the probability vector of the uniform distribution, which is used to approximate the classification probability of OOD data.

### 4. Comprehensive Scoring Function

To comprehensively evaluate the distribution differences, the author proposes a comprehensive scoring function \(S(x)\), which combines the measurement results of the feature and probability spaces:
\[
S(x)=\lambda / \epsilon_{KL}(x, \hat{x})+(1 - \lambda) / \epsilon_{\ell_2}(x, \hat{x})
\]
where \(\lambda\in[0, 1]\) is the balance coefficient.

### 5. Experimental Results

Through extensive experiments, the author proves that the D3 framework has achieved the state - of - the - art detection performance on a variety of OOD datasets, especially performing excellently in the False Positive Rate (FPR) and Area Under the Receiver Operating Characteristic Curve (AUROC) indicators.

In conclusion, this paper aims to propose a more effective and comprehensive OOD detection framework by re - thinking the evaluation objectives and measurement methods in DM - based OoD detection methods.