Abstract:The multispectral (MS) and the panchromatic (PAN) images belong to different modalities with specific advantageous properties. Therefore, there is a large representation gap between them. Moreover, the features extracted independently by the two branches belong to different feature spaces, which is not conducive to the subsequent collaborative classification. At the same time, different layers also have different representation capabilities for objects with large size differences. In order to dynamically and adaptively transfer the dominant attributes, reduce the gap between them, find the best shared layer representation, and fuse the features of different representation capabilities, this article proposes an adaptive migration collaborative network (AMC-Net) for multimodal remote-sensing (RS) images classification. First, for the input of the network, we combine principal component analysis (PCA) and nonsubsampled contourlet transformation (NSCT) to migrate the advantageous attributes of the PAN and the MS images to each other. This not only improves the quality of images themselves, but also increases the similarity between the two images, thereby reducing the representational gap between them and the pressure on the subsequent classification network. Second, for the interaction on the feature migrate branch, we design a feature progressive migration fusion unit (FPMF-Unit) based on the adaptive cross-stitch unit of correlation coefficient analysis (CCA), which can make the network automatically learn the features that need to be shared and migrated, aiming to find the best shared-layer representation for multifeature learning. And we design an adaptive layer fusion mechanism module (ALFM-Module), which can adaptively fuse features of different layers, aiming to clearly model the dependencies among multiple layers for different sized objects. Finally, for the output of the network, we add the calculation of the correlation coefficient to the loss function, which can make the network converge to the global optimum as much as possible. The experimental results indicate that AMC-Net can achieve competitive performance. And the code for the network framework is available at: https://github.com/ru-willow/A-AFM-ResNet.

MAMInet II: Illumination Insensitive Modalitywise Assimilation Guided Multistage Interaction Network for Hyperspectral and LiDAR Joint Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Remote Sensing Collaborative Classification Using Multimodal Adaptive Modulation Network

AMIANet: Asymmetric Multimodal Interactive Augmentation Network for Semantic Segmentation of Remote Sensing Imagery

HI2D2FNet: Hyperspectral Intrinsic Image Decomposition Guided Data Fusion Network for Hyperspectral and LiDAR Classification

MS2CANet: Multiscale Spatial–Spectral Cross-Modal Attention Network for Hyperspectral Image and LiDAR Classification

Multilevel Attention Dynamic-Scale Network for HSI and LiDAR Data Fusion Classification

Crossmodal Sequential Interaction Network for Hyperspectral and LiDAR Data Joint Classification

Multiple Information Collaborative Fusion Network for Joint Classification of Hyperspectral and LiDAR Data

MSLAENet: Multiscale Learning and Attention Enhancement Network for Fusion Classification of Hyperspectral and LiDAR Data

Joint Contextual Representation Model-Informed Interpretable Network with Dictionary Aligning for Hyperspectral and LiDAR Classification

A Multi-Scale Pseudo-Siamese Network with an Attention Mechanism for Classification of Hyperspectral and LiDAR Data.

Dual-Branch Dynamic Modulation Network for Hyperspectral and LiDAR Data Classification.

An Adaptive Migration Collaborative Network for Multimodal Image Classification.

A Collaborative Correlation-Matching Network for Multimodality Remote Sensing Image Classification.

Multimodal Semantic Collaborative Classification for Hyperspectral Images and LiDAR Data

Hyperspectral and SAR Image Classification via Multiscale Interactive Fusion Network

A Multi-Scale Progressive Collaborative Attention Network for Remote Sensing Fusion Classification.

MBSI-Net: Multimodal Balanced Self-Learning Interaction Network for Image Classification

AM3Net: Adaptive Mutual-learning-based Multimodal Data Fusion Network

Joint Classification of Hyperspectral and LiDAR Data Based on Position-Channel Cooperative Attention Network