Abstract:Local-feature-based Few-Shot Learning (FSL) has attracked a lot of attention and achieved great progress recently. Given an image, the model extracts a group of local features through the Fully Convolutional Network (FCN), each of which contains information from the corresponding receptive field of the image. The challenging problem is that how to exploit the local-feature-level similarities to generate the image-level similarity. Towards this, many existing works have proposed different heuristic rules or settings. In this paper, we first follow existing works and systematically propose several modified methods for local feature matching, induced by a novel and improved heterogeneous matching mechanism. However, these heuristic methods are not optimal to highlight the most informative local feature pairs to represent the image-level similarity, and also can not generalize well to different tasks. Therefore, we propose a new idea called Sinkhorn Metrics (SM). We consider the local-feature-based FSL as the Regularized Optimal Transport (ROT) problem. The cost matrix is formed by the similarities of local feature pairs. The marginals indicating the importance of each local feature are obtained by a new attentive cross-comparison module. The optimal transportation plan is used as weights to aggregate all the local-feature-level similarities to obtain the image-level similarity. We exploit the Sinkhorn algorithm to solve the ROT problem, which is efficient for the end-to-end training. We conduct a hybrid experiment on SM with some heuristic baselines to demonstrate its compatibility. Extensive ablation studies are performed to fully evaluate important hyper-parameters and settings. Our method achieves a series of state-of-the-arts on multiple datasets in both the single-domain and cross-domain FSL scenarios (The code for evaluation, trained model, and datasets in this study are available at https://github.com/Wangduo428/few-shot-learning-SM).

Global-Local Interplay in Semantic Alignment for Few-Shot Learning

Local Spatial Alignment Network for Few-Shot Learning

Local-Global Semantic Fusion Single-shot Classification Method.

Collect and Select: Semantic Alignment Metric Learning for Few-Shot Learning

Imposing Semantic Consistency of Local Descriptors for Few-Shot Learning

LGSim: local task-invariant and global task-specific similarity for few-shot classification

Global- and Local-aware Feature Augmentation with Semantic Orthogonality for Few-shot Image Classification

Local Feature Semantic Alignment Network for Few-Shot Image Classification

Learning to Compare Relation: Semantic Alignment for Few-Shot Learning

Global–Local Query-Support Cross-Attention for Few-Shot Semantic Segmentation

BDLA: Bi-directional local alignment for few-shot learning

Global-local prompts guided image-text embedding, alignment and aggregation for multi-label zero-shot learning

Revisiting Local and Global Descriptor-Based Metric Network for Few-Shot SAR Target Classification

Local descriptor-based spatial cross attention network for few-shot learning

Saliency-Guided Mutual Learning Network for Few-shot Fine-grained Visual Recognition

Few-Shot Learning With Global Class Representations

Only Image Cosine Embedding for Few-Shot Learning

Improved local-feature-based few-shot learning with Sinkhorn metrics

Tailoring Embedding Function to Heterogeneous Few-Shot Tasks by Global and Local Feature Adaptors

Feature Aligning Few shot Learning Method Using Local Descriptors Weighted Rules

SpatialFormer: Semantic and Target Aware Attentions for Few-Shot Learning