Abstract:Unsupervised domain adaptation has limitations when encountering label discrepancy between the source and target domains. While open-set domain adaptation approaches can address situations when the target domain has additional categories, these methods can only detect them but not further classify them. In this paper, we focus on a more challenging setting dubbed Domain Adaptive Zero-Shot Learning (DAZSL), which uses semantic embeddings of class tags as the bridge between seen and unseen classes to learn the classifier for recognizing all categories in the target domain when only the supervision of seen categories in the source domain is available. The main challenge of DAZSL is to perform knowledge transfer across categories and domain styles simultaneously. To this end, we propose a novel end-to-end learning mechanism dubbed Three-way Semantic Consistent Embedding (TSCE) to embed the source domain, target domain, and semantic space into a shared space. Specifically, TSCE learns domain-irrelevant categorical prototypes from the semantic embedding of class tags and uses them as the pivots of the shared space. The source domain features are aligned with the prototypes via their supervised information. On the other hand, the mutual information maximization mechanism is introduced to push the target domain features and prototypes towards each other. By this way, our approach can align domain differences between source and target images, as well as promote knowledge transfer towards unseen classes. Moreover, as there is no supervision in the target domain, the shared space may suffer from the catastrophic forgetting problem. Hence, we further propose a ranking-based embedding alignment mechanism to maintain the consistency between the semantic space and the shared space. Experimental results on both I2AwA and I2WebV clearly validate the effectiveness of our method. Code is available at https://github.com/tiggers23/TSCE-Domain-Adaptive-Zero-Shot-Learning.

Semantic Level Image Retrieval in Domain Adaptive Semantic Segmentation

Domain-Specific Modeling and Semantic Alignment for Image-Based 3d Model Retrieval

When Masked Image Modeling Meets Source-free Unsupervised Domain Adaptation: Dual-Level Masked Network for Semantic Segmentation

Domain Adaptation for Remote Sensing Image Semantic Segmentation: An Integrated Approach of Contrastive Learning and Adversarial Learning

Unsupervised Domain Adaptation for Referring Semantic Segmentation

Image Retrieval Based on Fuzzy Semantic Relevance Matrix

Semantic Concentration for Domain Adaptation

Category Re-Balancing Based Domain Adaptive Semantic Segmentation

Unsupervised Domain Adaptation for Remote Sensing Image Semantic Segmentation Using Region and Category Adaptive Domain Discriminator

Context-Aware Domain Adaptation in Semantic Segmentation

A Domain Adaptive Semantic Segmentation Method Using Contrastive Learning and Data Augmentation

Partial Domain Adaptation on Semantic Segmentation

Unsupervised Domain Adaptation for Semantic Segmentation of High-resolution Remote Sensing Imagery Driven by Category-Certainty Attention

Adversarial Domain Adaptation with Semantic Consistency for Cross-Domain Image Classification

Unsupervised Domain Adaptive Point Cloud Semantic Segmentation.

Semantic Consistent Embedding for Domain Adaptive Zero-Shot Learning

Learning Semantic Representations for Unsupervised Domain Adaptation.

Knowledge Based Domain Adaptation for Semantic Segmentation.

Unsupervised Domain Adaptation Multi-Level Adversarial Network for Semantic Segmentation Based on Multi-Modal Features

Unsupervised Domain Adaptation for Point Cloud Semantic Segmentation via Graph Matching