Abstract:Prototypical Network is a popular few-shot solver that aims at establishing a feature metric generalizable to novel few-shot classification (FSC) tasks using deep neural networks. However, its performance drops dramatically when generalizing to the FSC tasks in new domains. In this study, we revisit this problem and argue that the devil lies in the simplicity bias pitfall in neural networks. In specific, the network tends to focus on some biased shortcut features (e.g., color, shape, etc.) that are exclusively sufficient to distinguish very few classes in the meta-training tasks within a pre-defined domain, but fail to generalize across domains as some desirable semantic features. To mitigate this problem, we propose a Local-global Distillation Prototypical Network (LDP-net). Different from the standard Prototypical Network, we establish a two-branch network to classify the query image and its random local crops, respectively. Then, knowledge distillation is conducted among these two branches to enforce their class affiliation consistency. The rationale behind is that since such global-local semantic relationship is expected to hold regardless of data domains, the local-global distillation is beneficial to exploit some cross-domain transferable semantic features for feature metric establishment. Moreover, such local-global semantic consistency is further enforced among different images of the same class to reduce the intra-class semantic variation of the resultant feature. In addition, we propose to update the local branch as Exponential Moving Average (EMA) over training episodes, which makes it possible to better distill cross-episode knowledge and further enhance the generalization performance. Experiments on eight cross-domain FSC benchmarks empirically clarify our argument and show the state-of-the-art results of LDP-net. Code is available in https://github.com/NWPUZhoufei/LDP-Net

Relevance equilibrium network for cross-domain few-shot learning

Reweighting and Information-Guidance Networks for Few-Shot Learning

Free-Lunch for Cross-Domain Few-Shot Learning: Style-Aware Episodic Training with Robust Contrastive Learning

Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey

Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition.

Wave-SAN: Wavelet based Style Augmentation Network for Cross-Domain Few-Shot Learning

Task Context Transformer and GCN for Few-Shot Learning of Cross-Domain

Cross-Domain Cross-Set Few-Shot Learning via Learning Compact and Aligned Representations

Gradient-Guided Channel Masking for Cross-Domain Few-Shot Learning

Target Oriented Dynamic Adaption for Cross-Domain Few-Shot Learning

Spectral Decomposition and Transformation for Cross-domain Few-shot Learning

TACDFSL: Task Adaptive Cross Domain Few-Shot Learning

Revisiting Prototypical Network for Cross Domain Few-Shot Learning

MemREIN: Rein the Domain Shift for Cross-Domain Few-Shot Learning.

Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty

Meta-Collaborative Comparison for Effective Cross-Domain Few-Shot Learning

ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning

A Few-shot-learning-based Method to Object Recognition in Multiple Scenarios.

Self-taught cross-domain few-shot learning with weakly supervised object localization and task-decomposition

Knowledge Transduction for Cross-Domain Few-Shot Learning