Abstract:With the development of remote sensing technology, more and more data sources are available for landcover classification tasks, such as hyperspectral images (HSIs), light detection and ranging (LiDAR) data, and synthetic aperture radar (SAR) data. Due to the unique information carried by different sources, the collaborative use of multiple remote sensing data has become a key research direction in landcover classification tasks. Most existing methods are only capable of dealing with two types of remote-sensing images, limiting the potential for the collaboration of more sources. Also, the traditional feature fusion method uses addition or concatenation manner to integrate information, which makes it difficult to make full use of complement characteristics between modalities. As a remedy, we propose a novel multimodal adaptive modulation network (MAMNet), for landcover classification tasks using multimodal remote sensing data. First, the cross-modal interacting module (CIM) is utilized for information absorption between modalities. The feature representation is enhanced, and the modality-specific information is preserved. Second, the modal attention layer (MAL) is designed for multimodal feature fusion. Softmax attention is utilized to eliminate redundant information among the three modal features. Finally, an adaptive multimodal margin loss (AMM loss) is proposed to balance the consistency and diversity of multimodal features. It encourages adjustable decision margins between sources, which enables the model to better utilize complementary information between modalities and partially avoids model-overfitting by defining a more difficult learning target. Experimental results on two benchmark remote sensing datasets show the effectiveness of the proposed method compared with several state-of-the-art approaches.

Classification of Remote Sensing Images Based on Band Selection and Multi-mode Feature Fusion.

A Novel Feature Fusion Approach for VHR Remote Sensing Image Classification

Multi-Feature Remote Sensing Image Classification Based on Data Fusion

A Novel MRF-Based Multifeature Fusion for Classification of Remote Sensing Images

A Novel Remote Sensing Image Classification Scheme Based on Data Fusion, Multiple Features and Ensemble Learning

Remote Sensing Scene Classification Using Heterogeneous Feature Extraction and Multi-Level Fusion

RS-MSSF Frame: Remote Sensing Image Classification Based on Extraction and Fusion of Multiple Spectral-Spatial Features

A Spatial-Spectral Bilinear Representation Fusion Network for Multimodal Classification

A multispectral remote sensing image fusion method based on classification learning

Remote Sensing Image Classification with Multiple Classifiers Based on Support Vector Machines

Remote Sensing Image Scene Classification Based on Fusion Method

Few-shot remote sensing scene classification based on multi subband deep feature fusion

Remote Sensing Collaborative Classification Using Multimodal Adaptive Modulation Network

Multi-source Remote Sensing Image Fusion Based on Support Vector Machine

A Multiscale Feature Fusion Approach for Classification of Very High Resolution Satellite Imagery Based on Wavelet Transform

Classifying Multisensor Images by Support Vector Machine in Chongming Dongtan

Research on multisource remote sensing image classification algorithms based on image fusion and the EM-HMRF

Hyperspectral Remote Sensing Image Classification Based on Decision Level Fusion

Hyperspectral and SAR Image Classification via Multiscale Interactive Fusion Network

Classification of Land Cover from Remote Sensing Fused Image Based on ICA-SVM and D-S Evidence Theory

Learning SAR-Optical Cross Modal Features for Land Cover Classification