Abstract:Compositional zero-shot learning (CZSL) aims to recognize novel compositions of attributes and objects learned from seen compositions. Previous works disentangle attribute and object by extracting shared and exclusive parts between image pairs sharing the same attribute (object), as well as aligning them with pretrained word embeddings to improve unseen attribute-object recognition. Despite the significant achievements of existing efforts, they are hampered by three limitations: (1) the efficacy of disentanglement is compromised due to the influence of the background and the intricate entanglement of attribute with object in the same parts. (2) existing word embeddings fail to capture complex multimodal semantic information. (3) overconfidence exhibited by existing models in seen compositions hinders their generalization to novel compositions. Being aware of these, we propose a novel framework named Multimodal Large Language Model (MLLM) embeddings and attribute smoothing guided disentanglement (TRIDENT) for CZSL. First, we leverage feature adaptive aggregation modules to mitigate the impact of background, and utilize learnable condition masks to capture multigranularity features for disentanglement. Then, the last hidden states of MLLM are employed as word embeddings for their superior representation capabilities. Moreover, we propose attribute smoothing with auxiliary attributes generated by Large Language Model (LLM) for seen compositions, addressing the issue of overconfidence by encouraging the model to learn more attributes in one given composition. Extensive experiments demonstrate that TRIDENT achieves state-of-the-art performance on three benchmarks.

Retrieval-Augmented Primitive Representations for Compositional Zero-Shot Learning

MRSP: Learn Multi-Representations of Single Primitive for Compositional Zero-Shot Learning

Zero-Shot Learning with Generative Latent Prototype Model.

Compositional Zero-shot Learning Via Progressive Language-based Observations

Continual Compositional Zero-Shot Learning

Simple Primitives With Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-Shot Learning

Disentangling Before Composing: Learning Invariant Disentangled Features for Compositional Zero-Shot Learning

Contextual Interaction via Primitive-based Adversarial Training For Compositional Zero-shot Learning

Compositional Zero-Shot Learning with Contextualized Cues and Adaptive Contrastive Training

Learning to Embed Seen/Unseen Compositions based on Graph Networks

Reference-Limited Compositional Zero-Shot Learning

Anticipating Future Object Compositions without Forgetting

Leveraging MLLM Embeddings and Attribute Smoothing for Compositional Zero-Shot Learning

Attention Based Simple Primitives for Open World Compositional Zero-Shot Learning

Learning Conditional Attributes for Compositional Zero-Shot Learning

Ontology-guided Semantic Composition for Zero-Shot Learning

Cross-composition Feature Disentanglement for Compositional Zero-shot Learning

ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning

Zero-Shot Compositional Concept Learning

Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning

Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning