Abstract:Compositional zero-shot learning aims to recognize unseen state-object compositions by leveraging known primitives (state and object) during training. However, effectively modeling interactions between primitives and generalizing knowledge to novel compositions remains a perennial challenge. There are two key factors: object-conditioned and state-conditioned variance, i.e., the appearance of states (or objects) can vary significantly when combined with different objects (or states). For instance, the state "old" can signify a vintage design for a "car" or an advanced age for a "cat". In this paper, we argue that these variances can be mitigated by predicting composition categories based on pre-observed primitive. To this end, we propose Progressive Language-based Observations (PLO), which can dynamically determine a better observation order of primitives. These observations comprise a series of concepts or languages that allow the model to understand image content in a step-by-step manner. Specifically, PLO adopts pre-trained vision-language models (VLMs) to empower the model with observation capabilities. We further devise two variants: 1) PLO-VLM: a two-step method, where a pre-observing classifier dynamically determines the observation order of two primitives. 2) PLO-LLM: a multi-step scheme, which utilizes large language models (LLMs) to craft composition-specific prompts for step-by-step observing. Extensive ablations on three challenging datasets demonstrate the superiority of PLO compared with state-of-the-art methods, affirming its abilities in compositional recognition.

Learning the Compositional Domains for Generalized Zero-shot Learning

Cluster-based Contrastive Disentangling for Generalized Zero-Shot Learning

Learning to Separate Domains in Generalized Zero-Shot and Open Set Learning: a Probabilistic Perspective.

OntoZSL: Ontology-enhanced Zero-shot Learning

Domain segmentation and adjustment for generalized zero-shot learning

Learn More from Less: Generalized Zero-Shot Learning with Severely Limited Labeled Data

Compositional Zero-shot Learning Via Progressive Language-based Observations

Distinguishing Unseen from Seen for Generalized Zero-shot Learning

Improving Generalized Zero-Shot Learning by Semantic Discriminator

Less but Better: Enabling Generalized Zero-shot Learning Towards Unseen Domains by Intrinsic Learning from Redundant LLM Semantics

Disentangled Ontology Embedding for Zero-shot Learning

Transductive Unbiased Embedding for Zero-Shot Learning

A Review of Generalized Zero-Shot Learning Methods

A Boundary Based Out-of-Distribution Classifier for Generalized Zero-Shot Learning

Self-Supervised Domain-Aware Generative Network for Generalized Zero-Shot Learning

Alleviating Domain Shift Via Discriminative Learning for Generalized Zero-Shot Learning

Improving generalized zero-shot learning via cluster-based semantic disentangling representation

Meta-Transfer Networks for Zero-Shot Learning

Zero-Shot Learning with Generative Latent Prototype Model.

Domain-Specific Embedding Network for Zero-Shot Recognition

Semantics Disentangling for Generalized Zero-Shot Learning