Abstract:Vanilla CNNs, as uncalibrated classifiers, suffer from classifying out-of-distribution (OOD) samples nearly as confidently as in-distribution samples. To tackle this challenge, some recent works have demonstrated the gains of leveraging available OOD sets for training end-to-end calibrated CNNs. However, a critical question remains unanswered in these works: how to differentiate OOD sets for selecting the most effective one(s) that induce training such CNNs with high detection rates on unseen OOD sets? To address this pivotal question, we provide a criterion based on generalization errors of Augmented-CNN, a vanilla CNN with an added extra class employed for rejection, on in-distribution and unseen OOD sets. However, selecting the most effective OOD set by directly optimizing this criterion incurs a huge computational cost. Instead, we propose three novel computationally-efficient metrics for differentiating between OOD sets according to their "protection" level of in-distribution sub-manifolds. We empirically verify that the most protective OOD sets -- selected according to our metrics -- lead to A-CNNs with significantly lower generalization errors than the A-CNNs trained on the least protective ones. We also empirically show the effectiveness of a protective OOD set for training well-generalized confidence-calibrated vanilla CNNs. These results confirm that 1) all OOD sets are not equally effective for training well-performing end-to-end models (i.e., A-CNNs and calibrated CNNs) for OOD detection tasks and 2) the protection level of OOD sets is a viable factor for recognizing the most effective one. Finally, across the image classification tasks, we exhibit A-CNN trained on the most protective OOD set can also detect black-box FGS adversarial examples as their distance (measured by our metrics) is becoming larger from the protected sub-manifolds.

Adaptive Hierarchical Graph Cut for Multi-granularity Out-of-distribution Detection

Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection

AHA: Human-Assisted Out-of-Distribution Generalization and Detection

GOODAT: Towards Test-time Graph Out-of-Distribution Detection

Detecting Out-of-Distribution Examples Via Class-Conditional Impressions Reappearing

GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection

Toward Metrics for Differentiating Out-of-Distribution Sets

A Data-centric Framework to Endow Graph Neural Networks with Out-Of-Distribution Detection Ability.

HGOE: Hybrid External and Internal Graph Outlier Exposure for Graph Out-of-Distribution Detection

Improving Out-of-Distribution Detection with Disentangled Foreground and Background Features

Heterophilic Graph Invariant Learning for Out-of-Distribution of Fraud Detection

Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement

MultiCut-MultiMix: a Two-Level Data Augmentation Method for Detecting Small and Densely Distributed Objects in Large-Size Images

Mahalanobis-Aware Training for Out-of-Distribution Detection

GAIA: Delving into Gradient-based Attribution Abnormality for Out-of-distribution Detection

ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms

Are We Ready for Out-of-Distribution Detection in Digital Pathology?

Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Data

Low-Dimensional Gradient Helps Out-of-Distribution Detection

FCFDA: Fine–Coarse–Fine Progressive Graph Framework With Distribution Alignment for Hyperspectral Image Change Detection

Look Around and Find Out: OOD Detection with Relative Angles