Abstract:Zero-shot learning (ZSL) aims to recognize images of novel classes, but does not use any images belonging to the novel classes during model training, which is realized by exploiting the auxiliary semantic information. Recently, most ZSL methods focus on learning visual-semantic embeddings to transfer knowledge from the seen classes to the novel classes. Visual-semantic embedding is usually established based on the visual features of images and the semantic information of classes, i.e., class attributes. However, image features are extracted at the individual level, while class attributes are obtained at the group level, so the granularity of these features is different, which makes it difficult to match the two kinds of features. To tackle such problem, we propose hierarchical coupled discriminative dictionary learning (HCDDL) method to hierarchically establish visual-semantic embedding at class-level and image-level with a coarse-to-fine way. Firstly, a class-level coupled dictionary is trained to build basic and coarse-grained connection between visual space and semantic space. Using the class-level coupled dictionary, image attributes are generated. Based on the fine-grained image attributes and images features, an image-level coupled dictionary is learned. In addition, during the learning of hierarchical coupled dictionaries, the discriminative losses are adopted to ensure dictionaries learn more accurate representation, which is beneficial to the recognition task. Recognition of unseen images is performed through searching the class nearest to the unseen image in multiple spaces. Experiments on four widely used benchmark datasets show the effectiveness of the proposed method, and sufficient ablation experiments demonstrate that the coarse-to-fine way leads to good performances.

Heuristic Once Learning for Image & Text Duality Information Processing.

Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

Multimodal One-Shot Learning of Speech and Images

Learning Paired-associate Images with An Unsupervised Deep Learning Architecture

Dual-stream Multi-Modal Graph Neural Network for Few-Shot Learning

Learning from One and Only One Shot

Deep Multiple Instance Learning for Zero-Shot Image Tagging

Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models

Unified Contrastive Learning in Image-Text-Label Space

Multi-Level Semantic Feature Augmentation for One-Shot Learning

Once Learning for Looking and Identifying Based on YOLO-v5 Object Detection.

One-Shot Learning for Language Modelling

DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations

Unsupervised One-shot Learning of Both Specific Instances and Generalised Classes with a Hippocampal Architecture

Language-Driven Cross-Modal Classifier for Zero-Shot Multi-Label Image Recognition

DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations

Multimodal Prototypical Networks for Few-shot Learning

Multi-Level Similarity Learning for Low-Shot Recognition

Hierarchical Coupled Discriminative Dictionary Learning for Zero-Shot Learning

DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment

Dual Learning: Theoretical Study and an Algorithmic Extension