Few-Shot Open-Set Collaborative Classification of Multispectral and Hyperspectral Images with Adaptive Joint Similarity Metric

Bin Guo,Xiangrong Zhang,Tianzhu Liu,Yanfeng Gu
DOI: https://doi.org/10.1109/tgrs.2024.3502236
IF: 8.2
2024-01-01
IEEE Transactions on Geoscience and Remote Sensing
Abstract:Hyperspectral (HS) images have higher spectral resolution than multispectral (MS) images, but they have a narrower swath than MS images. The limited spectral resolution of MS images constrains their classification capabilities, and annotating remote sensing data is time-consuming and laborious. Additionally, large-scale MS images may contain unknown classes not present in the training data. This paper attempts to use partially overlapping HS images with limited labels to assist in the classification of large-scene MS images. It can correctly distinguish known classes and simultaneously identify unknown classes, thereby achieving better classification results for MS images. To address this challenge, a few-shot open-set hyperspectral-multispectral (HS-MS) image collaborative classification method is proposed. Specifically, a spectral-spatial feature interactive enhancement (SSFIE) module is designed for richer feature extraction and enhanced classification capabilities in the feature extraction stage. In the few-shot learning (FSL) stage, an adaptive joint similarity metric criterion is proposed to improve feature mapping between the source and target domains. Discriminative joint probability adaptation is used for domain adaptation and to enhance feature discriminability, while batch nuclear-norm maximization (BNM) is employed to increase the feature diversity. In the testing phase, the open-set classification module is designed to correctly classify samples of known classes while simultaneously distinguishing unknown classes. Experimental results on four cross-domain HS-MS data pairs demonstrate that our proposed method outperforms state-of-the-art methods.
What problem does this paper attempt to address?