Abstract:Compositional zero-shot learning aims to recognize unseen state-object compositions by leveraging known primitives (state and object) during training. However, effectively modeling interactions between primitives and generalizing knowledge to novel compositions remains a perennial challenge. There are two key factors: object-conditioned and state-conditioned variance, i.e., the appearance of states (or objects) can vary significantly when combined with different objects (or states). For instance, the state "old" can signify a vintage design for a "car" or an advanced age for a "cat". In this paper, we argue that these variances can be mitigated by predicting composition categories based on pre-observed primitive. To this end, we propose Progressive Language-based Observations (PLO), which can dynamically determine a better observation order of primitives. These observations comprise a series of concepts or languages that allow the model to understand image content in a step-by-step manner. Specifically, PLO adopts pre-trained vision-language models (VLMs) to empower the model with observation capabilities. We further devise two variants: 1) PLO-VLM: a two-step method, where a pre-observing classifier dynamically determines the observation order of two primitives. 2) PLO-LLM: a multi-step scheme, which utilizes large language models (LLMs) to craft composition-specific prompts for step-by-step observing. Extensive ablations on three challenging datasets demonstrate the superiority of PLO compared with state-of-the-art methods, affirming its abilities in compositional recognition.

Zero-Shot Compositional Concept Learning

Joint Learning of Attended Zero-Shot Features and Visual-Semantic Mapping.

Learning Attention as Disentangler for Compositional Zero-shot Learning

Compositional Zero-Shot Learning with Contextualized Cues and Adaptive Contrastive Training

Compositional Zero-shot Learning Via Progressive Language-based Observations

Meta-Transfer Networks for Zero-Shot Learning

Learning Conditional Attributes for Compositional Zero-Shot Learning

Continual Compositional Zero-Shot Learning

Reference-Limited Compositional Zero-Shot Learning

Ontology-guided Semantic Composition for Zero-Shot Learning

A causal view of compositional zero-shot recognition

C2C: Component-to-Composition Learning for Zero-Shot Compositional Action Recognition

OntoZSL: Ontology-enhanced Zero-shot Learning

ZeroC: A Neuro-Symbolic Model for Zero-shot Concept Recognition and Acquisition at Inference Time

Multi-modal Generative Adversarial Network for Zero-Shot Learning

Anticipating Future Object Compositions without Forgetting

Agree to Disagree: Exploring Partial Semantic Consistency against Visual Deviation for Compositional Zero-Shot Learning

Learning Graph Embeddings for Open World Compositional Zero-Shot Learning

Hybrid Discriminative Attribute-Object Embedding Network for Compositional Zero-Shot Learning

Retrieval-Augmented Primitive Representations for Compositional Zero-Shot Learning

Attention Based Simple Primitives for Open World Compositional Zero-Shot Learning