Abstract:Zero-Shot Learning (ZSL) focuses on classifying samples of unseen classes with only their side semantic information presented during training. It cannot handle real-life, open-world scenarios where there are test samples of unknown classes for which neither samples (e.g., images) nor their side semantic information is known during training. Open-Set Recognition (OSR) is dedicated to addressing the unknown class issue, but existing OSR methods are not designed to model the semantic information of the unseen classes. To tackle this combined ZSL and OSR problem, we consider the case of "Zero-Shot Open-Set Recognition" (ZS-OSR), where a model is trained under the ZSL setting but it is required to accurately classify samples from the unseen classes while being able to reject samples from the unknown classes during inference. We perform large experiments on combining existing state-of-the-art ZSL and OSR models for the ZS-OSR task on four widely used datasets adapted from the ZSL task, and reveal that ZS-OSR is a non-trivial task as the simply combined solutions perform badly in distinguishing the unseen-class and unknown-class samples. We further introduce a novel approach specifically designed for ZS-OSR, in which our model learns to generate adversarial semantic embeddings of the unknown classes to train an unknowns-informed ZS-OSR classifier. Extensive empirical results show that our method 1) substantially outperforms the combined solutions in detecting the unknown classes while retaining the classification accuracy on the unseen classes and 2) achieves similar superiority under generalized ZS-OSR settings. Our code is available at https://github.com/lhrst/ASE .

Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks

Joint Learning of Attended Zero-Shot Features and Visual-Semantic Mapping.

GENERATING MANIFOLD-ALIGNED SEMANTIC FEATURE FOR ZERO-SHOT LEARNING

Zero-Shot Learning With Attentive Region Embedding and Enhanced Semantics

Multi-modal Generative Adversarial Network for Zero-Shot Learning

Attentive Semantic Preservation Network for Zero-Shot Learning.

Zero-Shot Learning via Discriminative Dual Semantic Auto-Encoder

Zero-Shot Learning via Structure-Aligned Generative Adversarial Network

OntoZSL: Ontology-enhanced Zero-shot Learning

Domain-Specific Embedding Network for Zero-Shot Recognition

Learning discriminative visual semantic embedding for zero-shot recognition

SR-GAN: Semantic Rectifying Generative Adversarial Network for Zero-shot Learning

Semantic Autoencoder for Zero-Shot Learning

Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

Bi-Adversarial Auto-Encoder for Zero-Shot Learning

Zero-Shot Object Detection by Semantics-Aware DETR with Adaptive Contrastive Loss.

Learning complementary semantic information for zero-shot recognition

Learning adversarial semantic embeddings for zero-shot recognition in open worlds

Zero-Shot Learning on Semantic Class Prototype Graph

Towards Zero-Shot Learning: A Brief Review and an Attention-Based Embedding Network