Abstract:Most existing zero-shot learning approaches exploit transfer learning via an intermediate semantic representation shared between an annotated auxiliary dataset and a target dataset with different classes and no annotation. A projection from a low-level feature space to the semantic representation space is learned from the auxiliary dataset and applied without adaptation to the target dataset. In this paper we identify two inherent limitations with these approaches. First, due to having disjoint and potentially unrelated classes, the projection functions learned from the auxiliary dataset/domain are biased when applied directly to the target dataset/domain. We call this problem the projection domain shift problem and propose a novel framework, transductive multi-view embedding, to solve it. The second limitation is the prototype sparsity problem which refers to the fact that for each target class, only a single prototype is available for zero-shot learning given a semantic representation. To overcome this problem, a novel heterogeneous multi-view hypergraph label propagation method is formulated for zero-shot learning in the transductive embedding space. It effectively exploits the complementary information offered by different semantic representations and takes advantage of the manifold structures of multiple representation spaces in a coherent manner. We demonstrate through extensive experiments that the proposed approach (1) rectifies the projection shift between the auxiliary and target domains, (2) exploits the complementarity of multiple semantic representations, (3) significantly outperforms existing methods for both zero-shot and N-shot recognition on three image and video benchmark datasets, and (4) enables novel cross-view annotation tasks.

An Inverse Mapping With Manifold Alignment For Zero-Shot Learning

GENERATING MANIFOLD-ALIGNED SEMANTIC FEATURE FOR ZERO-SHOT LEARNING

Joint Learning of Attended Zero-Shot Features and Visual-Semantic Mapping.

Learning Visually Aligned Semantic Graph for Cross-Modal Manifold Matching.

Manifold Embedding for Zero-Shot Recognition

Zero-Shot Recognition Using Dual Visual-Semantic Mapping Paths.

Matrix Tri-Factorization With Manifold Regularizations For Zero-Shot Learning

Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

Manifold Regularized Cross-Modal Embedding for Zero-Shot Learning

A Novel Perspective to Zero-shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion

Zero-Shot Leaning With Manifold Embedding

Zero-Shot Learning on Semantic Class Prototype Graph

Indirect visual–semantic alignment for generalized zero-shot recognition

Transductive Unbiased Embedding for Zero-Shot Learning

Learning discriminative visual semantic embedding for zero-shot recognition

Meta-Transfer Networks for Zero-Shot Learning

Transductive multi-view zero-shot learning

Zero-Shot Learning via Discriminative Dual Semantic Auto-Encoder

Vocabulary-Informed Zero-Shot and Open-Set Learning

Learning Latent Semantic Attributes for Zero-Shot Object Detection.

Transductive Zero-Shot Learning with a Self-Training Dictionary Approach