Abstract:Compositional zero-shot learning (CZSL) task aims to recognize unseen compositional visual concepts, e.g., sliced tomatoes, where the model is learned only from the seen compositions, e.g., sliced potatoes and red tomatoes. Thanks to the prompt tuning on large pre-trained visual language models such as CLIP, recent literature shows impressively better CZSL performance than traditional vision-based methods. However, the key aspects that impact the generalization to unseen compositions, including the diversity and informativeness of class context, and the entanglement between visual primitives, i.e., state and object, are not properly addressed in existing CLIP-based CZSL literature. In this paper, we propose a model by prompting the language-informed distribution, aka., PLID, for the CZSL task. Specifically, the PLID leverages pre-trained large language models (LLM) to (i) formulate the language-informed class distributions which are diverse and informative, and (ii) enhance the compositionality of the class embedding. Moreover, a visual-language primitive decomposition (VLPD) module is proposed to dynamically fuse the classification decisions from the compositional and the primitive space. Orthogonal to the existing literature of soft, hard, or distributional prompts, our method advocates prompting the LLM-supported class distributions, leading to a better zero-shot generalization. Experimental results on MIT-States, UT-Zappos, and C-GQA datasets show the superior performance of the PLID to the prior arts. Our code and models are released: <a class="link-external link-https" href="https://github.com/Cogito2012/PLID" rel="external noopener nofollow">this https URL</a>.

Learning Conditional Prompt for Compositional Zero-Shot Learning

Hierarchical Prompt Learning for Compositional Zero-Shot Recognition.

Prompting Language-Informed Distribution for Compositional Zero-Shot Learning

Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning

Compositional Zero-Shot Learning with Contextualized Cues and Adaptive Contrastive Training

Learning Conditional Attributes for Compositional Zero-Shot Learning

Prompting Large Pre-trained Vision-Language Models For Compositional Concept Learning

Compositional Zero-shot Learning Via Progressive Language-based Observations

DRPT: Disentangled and Recurrent Prompt Tuning for Compositional Zero-Shot Learning

GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning

Dual-Stream Contrastive Learning for Compositional Zero-Shot Recognition

Continual Compositional Zero-Shot Learning

Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

Learning to Embed Seen/Unseen Compositions based on Graph Networks

Learning Attention as Disentangler for Compositional Zero-shot Learning

Visual Primitives As Words: Alignment and Interaction for Compositional Zero-Shot Learning

Prompt Tuning for Zero-shot Compositional Learning

Anticipating Future Object Compositions without Forgetting

Reference-Limited Compositional Zero-Shot Learning

LVAR-CZSL: Learning Visual Attributes Representation for Compositional Zero-Shot Learning

MRSP: Learn Multi-Representations of Single Primitive for Compositional Zero-Shot Learning