Abstract:Few-shot open-set recognition, as a new paradigm, leveraging a limited amount of supervised data to identify specific Remote Sensing (RS) scene categories and generalize to novel ones. However, the data bias induced by the small sample size not only causes severe overfitting within base classes, but also impairs the capacity for inference to identify RS scenes in hitherto unobserved categories. Furthermore, owing to environmental influences, RS images frequently manifest notable intra-class disparities and comparatively low inter-class distinctions, intensifying the challenge in obtaining suitable classifiers. To address above issues, we investigate the utilization of a Multi-modal Foundational Model (MFM) infused with essential domain knowledge to mitigate the generalization limitations encountered in few-shot scenarios. Recognizing that existing MFMs with a visual-text dual-branch structure are primarily tailored for natural scenes, we propose a custom Frequency Distribution-based Multi-modal Fine-Tuning strategy (FreqDiMFT) in a parameter-efficient manner. More specifically, within the vision branch, we address the high inter-class similarity and intra-class diversity in RS images by embedding the local-global frequency distribution information to facilitate the recognition of RS scenes. To further amplify the model's generalization ability post transfer, we introduce an adaptive feature refinement module designed for Transformers, proficient in filtering redundant features resulting from domain disparities. To mitigate the domain drift on the textual branch, we adopt an input format that combines basic templates with domain expertise from RS end to generate more discriminative class prototypes. To fully verify the effectiveness of our FreqDiMFT in a more practical setting, we collect a Large-Scale hybrid dataset (LSRS). Extensive experiments demonstrate that, even with a scant number of training samples, our strategy yields advanced performances compared to state-of-the-art models.

RSCM: Region Selection and Concurrency Model for Multi-Class Weather Recognition.

A Multi-Task Framework for Weather Recognition

Extended Collaborative Representation-Based Hyperspectral Imagery Classification

A CNN-based Multi-Task Framework for Weather Recognition with Multi-Scale Weather Cues

Semantic Representation Fusion-Based Network for Robust Land Cover Classification in Foggy Conditions.

Two-Class Weather Classification

Multi-class Weather Classification Fusing Weather Dataset and Image Features

Improving Cloud Type Classification of Ground-Based Images Using Region Covariance Descriptors

Severe Convective Weather Classification In Remote Sensing Images By Semantic Segmentation

A CNN-RNN Architecture for Multi-Label Weather Recognition

Weighted multiscale region-level sparse representation for classification of high-spatial resolution remote sensing images

Weather-oriented Domain Generalization of Semantic Segmentation for Autonomous Driving.

A time, space and color-based classification of different weather conditions.

Modeling Weather Uncertainty for Multi-weather Co-Presence Estimation

A High-dimension Feature Spaces Clustering and Corresponding Weather Classification for Multi-spectral Satellite Images

Method of Weather Recognition Based on Decision-Tree-based SVM

A Collaborative Correlation-Matching Network for Multimodality Remote Sensing Image Classification.

A self-supervised deep-driven model for automatic weather classification from remote sensing images

Frequency-Aware Multi-Modal Fine-Tuning for Few-Shot Open-Set Remote Sensing Scene Classification

Multi-View Ground-Based Cloud Recognition by Transferring Deep Visual Information