Abstract:In Few-Shot Learning (FSL), the objective is to correctly recognize new samples from novel classes with only a few available samples per class. Existing methods in FSL primarily focus on learning transferable knowledge from base classes by maximizing the information between feature representations and their corresponding labels. However, this approach may suffer from the "supervision collapse" issue, which arises due to a bias towards the base classes. In this paper, we propose a solution to address this issue by preserving the intrinsic structure of the data and enabling the learning of a generalized model for the novel classes. Following the InfoMax principle, our approach maximizes two types of mutual information (MI): between the samples and their feature representations, and between the feature representations and their class labels. This allows us to strike a balance between discrimination (capturing class-specific information) and generalization (capturing common characteristics across different classes) in the feature representations. To achieve this, we adopt a unified framework that perturbs the feature embedding space using two low-bias estimators. The first estimator maximizes the MI between a pair of intra-class samples, while the second estimator maximizes the MI between a sample and its augmented views. This framework effectively combines knowledge distillation between class-wise pairs and enlarges the diversity in feature representations. By conducting extensive experiments on popular FSL benchmarks, our proposed approach achieves comparable performances with state-of-the-art competitors. For example, we achieved an accuracy of 69.53% on the miniImageNet dataset and 77.06% on the CIFAR-FS dataset for the 5-way 1-shot task.

Feature Transformation for Few-Shot Learning

Improving Few-shot Text Classification via Pretrained Language Representations

Graph-based few-shot learning with transformed feature propagation and optimal class allocation

Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation

When Low Resource NLP Meets Unsupervised Language Model: Meta-Pretraining then Meta-Learning for Few-Shot Text Classification (Student Abstract)

Multi-scale Feature Modification Network for FewShot Image Classification

Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification.

Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning

Few-Shot Classification of Aerial Scene Images via Meta-Learning

A Deep Feature Transformation Method Based on Differential Vector for Few-Shot Learning

Learning Embedding Adaptation for Few-Shot Learning

Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning

Feature Mixture on Pre-Trained Model for Few-Shot Learning

Hybrid Consistency Training with Prototype Adaptation for Few-Shot Learning

Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions

Embedding enhancement with foreground feature alignment and primitive knowledge for few-shot learning

Cross Attention Network for Few-shot Classification

PrototypeFormer: Learning to Explore Prototype Relationships for Few-shot Image Classification

Three-Dimension Attention Mechanism and Self-Supervised Pretext Task for Augmenting Few-Shot Learning

LibFewShot: A Comprehensive Library for Few-Shot Learning

Balancing Feature Alignment and Uniformity for Few-Shot Classification.