Abstract:Prototypical Network is a popular few-shot solver that aims at establishing a feature metric generalizable to novel few-shot classification (FSC) tasks using deep neural networks. However, its performance drops dramatically when generalizing to the FSC tasks in new domains. In this study, we revisit this problem and argue that the devil lies in the simplicity bias pitfall in neural networks. In specific, the network tends to focus on some biased shortcut features (e.g., color, shape, etc.) that are exclusively sufficient to distinguish very few classes in the meta-training tasks within a pre-defined domain, but fail to generalize across domains as some desirable semantic features. To mitigate this problem, we propose a Local-global Distillation Prototypical Network (LDP-net). Different from the standard Prototypical Network, we establish a two-branch network to classify the query image and its random local crops, respectively. Then, knowledge distillation is conducted among these two branches to enforce their class affiliation consistency. The rationale behind is that since such global-local semantic relationship is expected to hold regardless of data domains, the local-global distillation is beneficial to exploit some cross-domain transferable semantic features for feature metric establishment. Moreover, such local-global semantic consistency is further enforced among different images of the same class to reduce the intra-class semantic variation of the resultant feature. In addition, we propose to update the local branch as Exponential Moving Average (EMA) over training episodes, which makes it possible to better distill cross-episode knowledge and further enhance the generalization performance. Experiments on eight cross-domain FSC benchmarks empirically clarify our argument and show the state-of-the-art results of LDP-net. Code is available in https://github.com/NWPUZhoufei/LDP-Net

Cross-Domain Few-Shot Classification via Dense-Sparse-Dense Regularization

Cross-Domain Few-Shot classification via class-shared and class-specific dictionaries

Soft Weight Pruning for Cross-Domain Few-Shot Learning with Unlabeled Target Data

Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification

Cross-domain Few-Shot Classification Through Feature Confusion

Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learning

Cross-Domain Few-Shot Classification Via Adversarial Task Augmentation

Adaptive Semantic Consistency for Cross-domain Few-shot Classification

Convert Cross-Domain Classification into Few-Shot Learning: A Unified Prompt-Tuning Framework for Unsupervised Domain Adaptation

Target Oriented Dynamic Adaption for Cross-Domain Few-Shot Learning

Attentive fine-grained recognition for cross-domain few-shot classification

Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation

Cross-Domain Few-Shot Classification based on Lightweight Res2Net and Flexible GNN

Cross-Domain Few-Shot Segmentation via Iterative Support-Query Correspondence Mining

Cross-Domain Few-Shot Classification via Inter-Source Stylization

TACDFSL: Task Adaptive Cross Domain Few-Shot Learning

Revisiting Prototypical Network for Cross Domain Few-Shot Learning

Adaptive Fine-Tuning Strategy for Few-Shot Learning

Spectral Decomposition and Transformation for Cross-domain Few-shot Learning

Cross-Domain Self-Taught Network for Few-Shot Hyperspectral Image Classification

The More, the Better? Active Silencing of Non-Positive Transfer for Efficient Multi-Domain Few-Shot Classification