Abstract:Zero-Shot Learning (ZSL), which aims at automatically recognizing unseen objects, is a promising learning paradigm to understand new real-world knowledge for machines continuously. Recently, the Knowledge Graph (KG) has been proven as an effective scheme for handling the zero-shot task with large-scale and non-attribute data. Prior studies always embed relationships of seen and unseen objects into visual information from existing knowledge graphs to promote the cognitive ability of the unseen data. Actually, real-world knowledge is naturally formed by multimodal facts. Compared with ordinary structural knowledge from a graph perspective, multimodal KG can provide cognitive systems with fine-grained knowledge. For example, the text description and visual content can depict more critical details of a fact than only depending on knowledge triplets. Unfortunately, this multimodal fine-grained knowledge is largely unexploited due to the bottleneck of feature alignment between different modalities. To that end, we propose a multimodal intensive ZSL framework that matches regions of images with corresponding semantic embeddings via a designed dense attention module and self-calibration loss. It makes the semantic transfer process of our ZSL framework learns more differentiated knowledge between entities. Our model also gets rid of the performance limitation of only using rough global features. We conduct extensive experiments and evaluate our model on large-scale real-world data. The experimental results clearly demonstrate the effectiveness of the proposed model in standard zero-shot classification tasks.

Compound Projection Learning for Bridging Seen and Unseen Objects

Joint Learning of Attended Zero-Shot Features and Visual-Semantic Mapping.

Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

Convolutional prototype learning for zero-shot recognition

Holistically Associated Transductive Zero-Shot Learning

Meta-Transfer Networks for Zero-Shot Learning

Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning.

Zero-shot Learning Via Shared-Reconstruction-Graph Pursuit

Unbiased Hybrid Generation Network for Zero-Shot Learning

Class-specific Synthesized Dictionary Model for Zero-Shot Learning

OntoZSL: Ontology-enhanced Zero-shot Learning

Multi-Label Zero-Shot Learning with Structured Knowledge Graphs

Zero-Shot Leaning With Manifold Embedding

Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning

Transductive Unbiased Embedding for Zero-Shot Learning

Manifold Regularized Cross-Modal Embedding for Zero-Shot Learning

Attribute self-representation steered by exclusive lasso for zero-shot learning

Recognizing Unseen Objects via Multimodal Intensive Knowledge Graph Propagation

Embarrassingly Easy Zero-Shot Image Recognition

Bridging the Projection Gap: Overcoming Projection Bias Through Parameterized Distance Learning

Transductive Visual-Semantic Embedding for Zero-shot Learning.