Abstract:Zero-shot learning (ZSL) tackles the novel class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Semantic knowledge is typically represented by attribute descriptions shared between different classes, which act as strong priors for localizing object attributes that represent discriminative region features, enabling significant and sufficient visual-semantic interaction for advancing ZSL. Existing attention-based models have struggled to learn inferior region features in a single image by solely using unidirectional attention, which ignore the transferable and discriminative attribute localization of visual features for representing the key semantic knowledge for effective knowledge transfer in ZSL. In this paper, we propose a cross attribute-guided Transformer network, termed TransZero++, to refine visual features and learn accurate attribute localization for key semantic knowledge representations in ZSL. Specifically, TransZero++ employs an attribute → visual Transformer sub-net (AVT) and a visual → attribute Transformer sub-net (VAT) to learn attribute-based visual features and visual-based attribute features, respectively. By further introducing feature-level and prediction-level semantical collaborative losses, the two attribute-guided transformers teach each other to learn semantic-augmented visual embeddings for key semantic knowledge representations via semantical collaborative learning. Finally, the semantic-augmented visual embeddings learned by AVT and VAT are fused to conduct desirable visual-semantic interaction cooperated with class semantic vectors for ZSL classification. Extensive experiments show that TransZero++ achieves the new state-of-the-art results on three golden ZSL benchmarks and on the large-scale ImageNet dataset. The project website is available at: https://shiming-chen.github.io/TransZero-pp/TransZero-pp.html.

Characterizing Hierarchical Semantic-Aware Parts with Transformers for Generalized Zero-Shot Learning

Transformer-Based Approach Via Contrastive Learning for Zero-Shot Detection.

DSPformer: Discovering Semantic Parts with Token Growth and Clustering for Zero-Shot Learning

Hierarchical Prototype Learning for Zero-Shot Recognition

TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

TransZero: Attribute-guided Transformer for Zero-Shot Learning

Exploiting Semantic Attributes for Transductive Zero-Shot Learning

OntoZSL: Ontology-enhanced Zero-shot Learning

Deep Representation of Hierarchical Semantic Attributes for Zero-shot Learning

Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning

Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation

Meta-Transfer Networks for Zero-Shot Learning

Hybrid Routing Transformer for Zero-Shot Learning

Temporal–Semantic Aligning and Reasoning Transformer for Audio-Visual Zero-Shot Learning

Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning.

Transductive Unbiased Embedding for Zero-Shot Learning

HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning

Explanatory Object Part Aggregation for Zero-Shot Learning.

Holistically Associated Transductive Zero-Shot Learning

Transductive Multi-class and Multi-label Zero-shot Learning