Abstract:Compositional zero-shot learning (CZSL) aims to recognize novel compositions of attributes and objects learned from seen compositions. Previous works disentangle attribute and object by extracting shared and exclusive parts between image pairs sharing the same attribute (object), as well as aligning them with pretrained word embeddings to improve unseen attribute-object recognition. Despite the significant achievements of existing efforts, they are hampered by three limitations: (1) the efficacy of disentanglement is compromised due to the influence of the background and the intricate entanglement of attribute with object in the same parts. (2) existing word embeddings fail to capture complex multimodal semantic information. (3) overconfidence exhibited by existing models in seen compositions hinders their generalization to novel compositions. Being aware of these, we propose a novel framework named Multimodal Large Language Model (MLLM) embeddings and attribute smoothing guided disentanglement (TRIDENT) for CZSL. First, we leverage feature adaptive aggregation modules to mitigate the impact of background, and utilize learnable condition masks to capture multigranularity features for disentanglement. Then, the last hidden states of MLLM are employed as word embeddings for their superior representation capabilities. Moreover, we propose attribute smoothing with auxiliary attributes generated by Large Language Model (LLM) for seen compositions, addressing the issue of overconfidence by encouraging the model to learn more attributes in one given composition. Extensive experiments demonstrate that TRIDENT achieves state-of-the-art performance on three benchmarks.

CSCNET: Class-Specified Cascaded Network for Compositional Zero-Shot Learning

Hybrid Discriminative Attribute-Object Embedding Network for Compositional Zero-Shot Learning

Cluster-based Contrastive Disentangling for Generalized Zero-Shot Learning

Leveraging MLLM Embeddings and Attribute Smoothing for Compositional Zero-Shot Learning

Learning Conditional Attributes for Compositional Zero-Shot Learning

Compositional Zero-Shot Learning with Contextualized Cues and Adaptive Contrastive Training

MRSP: Learn Multi-Representations of Single Primitive for Compositional Zero-Shot Learning

Cross-composition Feature Disentanglement for Compositional Zero-shot Learning

Disentangled Ontology Embedding for Zero-shot Learning

Learning Attention as Disentangler for Compositional Zero-shot Learning

Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning

Simple Primitives With Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-Shot Learning

Continual Compositional Zero-Shot Learning

Retrieval-Augmented Primitive Representations for Compositional Zero-Shot Learning

Distilled Reverse Attention Network for Open-world Compositional Zero-Shot Learning

Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning

Contextual Interaction via Primitive-based Adversarial Training For Compositional Zero-shot Learning

Attention Based Simple Primitives for Open World Compositional Zero-Shot Learning

Class-specific Synthesized Dictionary Model for Zero-Shot Learning

OntoZSL: Ontology-enhanced Zero-shot Learning

Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning