Abstract:Few-shot classification aims to adapt to new tasks with limited labeled examples. To fully use the accessible data, recent methods explore suitable measures for the similarity between the query and support images and better high-dimensional features with meta-training and pre-training strategies. However, the potential of multi-modality information has barely been explored, which may bring promising improvement for few-shot classification. In this paper, we propose a Language-guided Prototypical Network (LPN) for few-shot classification, which leverages the complementarity of vision and language modalities via two parallel branches to improve the classifier. Concretely, to introduce language modality with limited samples in the visual task, we leverage a pre-trained text encoder to extract class-level text features directly from class names while processing images with a conventional image encoder. Then, we introduce a language-guided decoder to obtain text features corresponding to each image by aligning class-level features with visual features. Additionally, we utilize class-level features and prototypes to build a refined prototypical head, which generates robust prototypes in the text branch for follow-up measurement. Furthermore, we leverage the class-level features to align the visual features, capturing more class-relevant visual features. Finally, we aggregate the visual and text logits to calibrate the deviation of a single modality, enhancing the overall performance. Extensive experiments demonstrate the competitiveness of LPN against state-of-the-art methods on benchmark datasets.

Modified Prototypical Networks for Few-Shot Text Classification Based on Class-Covariance Metric and Attention

Hierarchical Attention Prototypical Networks for Few-Shot Text Classification.

Enhance Prototypical Network with Hybrid Loss for Few-Shot Text Classification

Few-Shot Text Classification with Edge-Labeling Graph Neural Network-Based Prototypical Network.

Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification.

Enhance Prototypical Network with Text Descriptions for Few-shot Relation Classification

Cross Attention Network for Few-shot Classification

Bimodal semantic fusion prototypical network for few-shot classification

Compositional Prototypical Networks for Few-Shot Classification

Improved prototypical networks for few-Shot learning

An Attention Mechanism-based Relation Network for Few-Shot Image Classification

Multi-scale Feature Modification Network for FewShot Image Classification

Distribution Consistency Based Covariance Metric Networks for Few-Shot Learning.

Bidirectional Matching Prototypical Network for Few-Shot Image Classification

Spatial Attention Network for Few-Shot Learning

Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification.

Contrastive prototype network with prototype augmentation for few-shot classification

Shared Nearest Neighbor Calibration for Few-Shot Classification.

Hybrid Attention-Based Prototypical Networks for Few-Shot Sound Classification

Hybrid Enhancement-based prototypical networks for few-shot relation classification

LPN: Language-guided Prototypical Network for few-shot classification