Abstract:Recently, zero-shot multi-label classification has garnered considerable attention for its capacity to operate predictions on unseen labels without human annotations. Nevertheless, prevailing approaches often use seen classes as imperfect proxies for unseen ones, resulting in suboptimal performance. Drawing inspiration from the success of text-to-image generation models in producing realistic images, we propose an innovative solution: generating synthetic data to construct a training set explicitly tailored for proxyless training on unseen labels. Our approach introduces a novel image generation framework that produces multi-label synthetic images of unseen classes for classifier training. To enhance diversity in the generated images, we leverage a pre-trained large language model to generate diverse prompts. Employing a pre-trained multi-modal CLIP model as a discriminator, we assess whether the generated images accurately represent the target classes. This enables automatic filtering of inaccurately generated images, preserving classifier accuracy. To refine text prompts for more precise and effective multi-label object generation, we introduce a CLIP score-based discriminative loss to fine-tune the text encoder in the diffusion model. Additionally, to enhance visual features on the target task while maintaining the generalization of original features and mitigating catastrophic forgetting resulting from fine-tuning the entire visual encoder, we propose a feature fusion module inspired by transformer attention mechanisms. This module aids in capturing global dependencies between multiple objects more effectively. Extensive experimental results validate the effectiveness of our approach, demonstrating significant improvements over state-of-the-art methods.

Text-Guided Diverse Image Synthesis for Long-Tailed Remote Sensing Object Classification

Diversified text-to-image generation via deep mutual information estimation

Tail Classes Matter: Long-Tailed Object Detection Revisited.

SAU: A Dual-Branch Network to Enhance Long-Tailed Recognition via Generative Models

Long-tailed Diffusion Models with Oriented Calibration

Long-Tailed Object Detection for Multimodal Remote Sensing Images

Text-Guided Mixup Towards Long-Tailed Image Categorization

Active Diversification of Head-Class Features in Bilateral-Expert Models for Enhanced Tail-Class Optimization in Long-Tailed Classification

LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Content

Text-guided Fourier Augmentation for long-tailed recognition

Mixed Mutual Transfer for Long-Tailed Image Classification

Adaptive Class Suppression Loss for Long-Tail Object Detection

Diverse and Tailored Image Generation for Zero-shot Multi-label Classification

ChatDiff: A ChatGPT-based diffusion model for long-tailed classification

Increasing Oversampling Diversity for Long-Tailed Visual Recognition.

Taming the Tail in Class-Conditional GANs: Knowledge Sharing via Unconditional Training at Lower Resolutions

A Dual Progressive Strategy for Long-Tailed Visual Recognition

Multi-expert contrastive learning for remote sensing long-tailed image classification

Enhanced multi-branch learning for long-tailed image recognition

Center-Wise Feature Consistency Learning for Long-Tailed Remote Sensing Object Recognition

Self Supervision to Distillation for Long-Tailed Visual Recognition