Abstract:Local-feature-based Few-Shot Learning (FSL) has attracked a lot of attention and achieved great progress recently. Given an image, the model extracts a group of local features through the Fully Convolutional Network (FCN), each of which contains information from the corresponding receptive field of the image. The challenging problem is that how to exploit the local-feature-level similarities to generate the image-level similarity. Towards this, many existing works have proposed different heuristic rules or settings. In this paper, we first follow existing works and systematically propose several modified methods for local feature matching, induced by a novel and improved heterogeneous matching mechanism. However, these heuristic methods are not optimal to highlight the most informative local feature pairs to represent the image-level similarity, and also can not generalize well to different tasks. Therefore, we propose a new idea called Sinkhorn Metrics (SM). We consider the local-feature-based FSL as the Regularized Optimal Transport (ROT) problem. The cost matrix is formed by the similarities of local feature pairs. The marginals indicating the importance of each local feature are obtained by a new attentive cross-comparison module. The optimal transportation plan is used as weights to aggregate all the local-feature-level similarities to obtain the image-level similarity. We exploit the Sinkhorn algorithm to solve the ROT problem, which is efficient for the end-to-end training. We conduct a hybrid experiment on SM with some heuristic baselines to demonstrate its compatibility. Extensive ablation studies are performed to fully evaluate important hyper-parameters and settings. Our method achieves a series of state-of-the-arts on multiple datasets in both the single-domain and cross-domain FSL scenarios (The code for evaluation, trained model, and datasets in this study are available at https://github.com/Wangduo428/few-shot-learning-SM).

Bilaterally Normalized Scale-Consistent Sinkhorn Distance for Few-Shot Image Classification

DBDC-SSL: Deep Brownian Distance Covariance With Self-Supervised Learning for Few-Shot Image Classification

Bidirectional Matching Prototypical Network for Few-Shot Image Classification

Improved local-feature-based few-shot learning with Sinkhorn metrics

Object-aware Long-short-range Spatial Alignment for Few-Shot Fine-Grained Image Classification

Multi-Similarity Enhancement Network for Few-Shot Segmentation.

Balancing Feature Alignment and Uniformity for Few-Shot Classification.

Local Spatial Alignment Network for Few-Shot Learning

Local Mutual Metric Network for Few-Shot Image Classification

LGSim: local task-invariant and global task-specific similarity for few-shot classification

Delve into Cosine: Few-shot Object Detection Via Adaptive Norm-cutting Scale

Multi-scale Feature Modification Network for FewShot Image Classification

Feature Aligning Few shot Learning Method Using Local Descriptors Weighted Rules

Category Alignment Mechanism for Few-Shot Image Classification

BSNet: Bi-Similarity Network for Few-shot Fine-grained Image Classification

Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification

Local Feature Semantic Alignment Network for Few-Shot Image Classification

Robust Network Based on Embedded Convolutional Block Attention Module and Earth Mover’s Distance for Few-Shot Learning

Few-Shot Image Classification Based on Swin Transformer + CSAM + EMD

Shared Nearest Neighbor Calibration for Few-Shot Classification.

Adaptive Local Feature Matching for Few-shot Fine-grained Image Recognition