Abstract:Open set recognition (OSR) constitutes a critical endeavor within the domain of computer vision, frequently deployed in applications, such as autonomous driving and medical imaging recognition. Existing OSR methodologies predominantly center on the acquisition of a profound association between image data and corresponding labels, facilitating the extraction of discriminative features instrumental for distinguishing novel categories. Nevertheless, real-world scenarios often introduce not only novel classes (referred to semantic shift) but also intricate environmental modifications that engender alterations in the distribution of established classes (termed as covariate shift). The latter phenomenon has the potential to undermine the robust correlation between images and labels established by conventional statistical correlation modeling approaches, consequently resulting in significant degradation of OSR performance. Causal correlation stands as the fundamental linkage between entities, routinely harnessed by humans to enhance their cognitive capacities for a more profound comprehension of the intricate world. With inspiration drawn from this perspective, our work herein introduces the causal inference-inspired open set recognition (CISOR) approach tailored for real-world OSR (RWOSR). CISOR represents the pioneering initiative to leverage the stability inherent in causal correlation to construct two pivotal modules: the covariate causal independence (CCI) module and the semantic causal uniqueness (SCU) module, both instrumental in addressing the RWOSR problem. The CCI module adeptly confronts the challenge of covariate shift by imposing constraints on the correlations between inter-class causal features. This strategy effectively mitigates the impact of spurious correlations between distinct categories on the generalization capacity of discriminative features. Furthermore, in order to counteract the issue of semantic shift, the SCU module harnesses correlations between causal features within the same class as constraints, thereby facilitating the extraction of resilient causal features endowed with superior discriminative capabilities. Empirical findings substantiate the superior efficacy of the proposed CIOSR method when compared to state-of-the-art approaches across diverse RWOSR benchmark datasets. The source code of this article will be available at https://github.com/yangzhen1252/RWOSR1.

Class-aware Variational Auto-encoder for Open Set Recognition

Conditional Gaussian Distribution Learning for Open Set Recognition

Exploring Diverse Representations for Open Set Recognition

Class-Specific Semantic Reconstruction for Open Set Recognition

Open Set Recognition using Vision Transformer with an Additional Detection Head

CAMV: Class Activation Mapping Value Towards Open Set Fine-Grained Recognition

Open Set Audio Classification Using Autoencoders Trained on Few Data

Latent Space Energy-based Model for Fine-grained Open Set Recognition

Learning Discriminative Feature Representation for Open Set Action Recognition

Contrastive Open Set Recognition

RD-OpenMax: Rethinking OpenMax for Robust Realistic Open-Set Recognition

GCL-OSDA: Uncertainty prediction-based graph collaborative learning for open-set domain adaptation

CVA2E: A Conditional Variational Autoencoder with an Adversarial Training Process for Hyperspectral Imagery Classification

Unified Classification and Rejection: A One-versus-All Framework

All Beings Are Equal in Open Set Recognition

Open-Vocabulary Object Detection with an Open Corpus

Adversarial Compact Wrapping Classifier Learning for Open Set Recognition

Towards Evidential and Class Separable Open Set Object Detection

Open Set Recognition in Real World

Deep Active Learning via Open Set Recognition

Know Yourself Better: Diverse Discriminative Feature Learning Improves Open Set Recognition