Abstract:Zero-shot learning (ZSL) is typically achieved by resorting to a class semantic embedding space to transfer the knowledge from the seen classes to unseen ones. Capturing the common semantic characteristics between the visual modality and the class semantic modality (e.g., attributes or word vector) is a key to the success of ZSL. In this paper, we propose a novel encoder-decoder approach, namely latent space encoding (LSE), to connect the semantic relations of different modalities. Instead of requiring a projection function to transfer information across different modalities like most previous work, LSE performs the interactions of different modalities via a feature aware latent space, which is learned in an implicit way. Specifically, different modalities are modeled separately but optimized jointly. For each modality, an encoder-decoder framework is performed to learn a feature aware latent space via jointly maximizing the recoverability of the original space from the latent space and the predictability of the latent space from the original space. To relate different modalities together, their features referring to the same concept are enforced to share the same latent codings. In this way, the common semantic characteristics of different modalities are generalized with the latent representations. Another property of the proposed approach is that it is easily extended to more modalities. Extensive experimental results on four benchmark datasets [animal with attribute, Caltech UCSD birds, aPY, and ImageNet] clearly demonstrate the superiority of the proposed approach on several ZSL tasks, including traditional ZSL, generalized ZSL, and zero-shot retrieval.

Ontology-guided Semantic Composition for Zero-Shot Learning

OntoZSL: Ontology-enhanced Zero-shot Learning

Joint Learning of Attended Zero-Shot Features and Visual-Semantic Mapping.

Disentangled Ontology Embedding for Zero-shot Learning

Compositional Zero-shot Learning Via Progressive Language-based Observations

Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification.

Semantic Graph for Zero-Shot Learning.

Meta-Transfer Networks for Zero-Shot Learning

Holistically Associated Transductive Zero-Shot Learning

Zero-Shot Learning Via Latent Space Encoding

Unified Framework for Open-World Compositional Zero-shot Learning

Generalized Zero-Shot Recognition based on Visually Semantic Embedding

Zero-Shot Compositional Concept Learning

Hierarchical Prompt Learning for Compositional Zero-Shot Recognition.

Zero-Shot Learning on Semantic Class Prototype Graph

Deep Representation of Hierarchical Semantic Attributes for Zero-shot Learning

Learning object-centric complementary features for zero-shot learning

Learning the Compositional Domains for Generalized Zero-shot Learning

Semantic-aware Visual Attributes Learning for Zero-Shot Recognition

Simple Primitives With Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-Shot Learning

Semantic Graph-enhanced Visual Network for Zero-shot Learning.