Abstract:Few-shot open-set recognition, as a new paradigm, leveraging a limited amount of supervised data to identify specific Remote Sensing (RS) scene categories and generalize to novel ones. However, the data bias induced by the small sample size not only causes severe overfitting within base classes, but also impairs the capacity for inference to identify RS scenes in hitherto unobserved categories. Furthermore, owing to environmental influences, RS images frequently manifest notable intra-class disparities and comparatively low inter-class distinctions, intensifying the challenge in obtaining suitable classifiers. To address above issues, we investigate the utilization of a Multi-modal Foundational Model (MFM) infused with essential domain knowledge to mitigate the generalization limitations encountered in few-shot scenarios. Recognizing that existing MFMs with a visual-text dual-branch structure are primarily tailored for natural scenes, we propose a custom Frequency Distribution-based Multi-modal Fine-Tuning strategy (FreqDiMFT) in a parameter-efficient manner. More specifically, within the vision branch, we address the high inter-class similarity and intra-class diversity in RS images by embedding the local-global frequency distribution information to facilitate the recognition of RS scenes. To further amplify the model's generalization ability post transfer, we introduce an adaptive feature refinement module designed for Transformers, proficient in filtering redundant features resulting from domain disparities. To mitigate the domain drift on the textual branch, we adopt an input format that combines basic templates with domain expertise from RS end to generate more discriminative class prototypes. To fully verify the effectiveness of our FreqDiMFT in a more practical setting, we collect a Large-Scale hybrid dataset (LSRS). Extensive experiments demonstrate that, even with a scant number of training samples, our strategy yields advanced performances compared to state-of-the-art models.

Multi-attention DeepEMD for Few-Shot Learning in Remote Sensing

Few-Shot Scene Classification With Multi-Attention Deepemd Network In Remote Sensing

Attention-Aware Deep Feature Embedding for Remote Sensing Image Scene Classification

Few-Shot Scene Classification with Attention Mechanism in Remote Sensing

A Novel Discriminative Enhancement Method for Few-Shot Remote Sensing Image Scene Classification

RS-MetaNet: Deep Metametric Learning for Few-Shot Remote Sensing Scene Classification

RS-SSKD: Self-Supervision Equipped with Knowledge Distillation for Few-Shot Remote Sensing Scene Classification

TAE-Net: Task-Adaptive Embedding Network for Few-Shot Remote Sensing Scene Classification

A Novel Deep Nearest Neighbor Neural Network for Few-Shot Remote Sensing Image Scene Classification

Task-Adaptive Embedding Learning with Dynamic Kernel Fusion for Few-Shot Remote Sensing Scene Classification

Few-shot learning for remote sensing image retrieval with maml

Few-Shot Remote Sensing Scene Classification Via Subspace Based on Multiscale Feature Learning

RS-MetaNet: Deep meta metric learning for few-shot remote sensing scene classification

Few-Shot Remote Sensing Image Scene Classification Based on Metric Learning and Local Descriptors

DLA-MatchNet for Few-Shot Remote Sensing Image Scene Classification

Frequency-Aware Multi-Modal Fine-Tuning for Few-Shot Open-Set Remote Sensing Scene Classification

Not Just Learning from Others but Relying on Yourself: A New Perspective on Few-Shot Segmentation in Remote Sensing

Multi-scale Self-attention-based Few-shot Object Detection for Remote Sensing Images.

DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learning

Few-Shot Scene Classification of Optical Remote Sensing Images Leveraging Calibrated Pretext Tasks

A Meta-Learning Framework for Few-Shot Classification of Remote Sensing Scene