Abstract:The multispectral (MS) and the panchromatic (PAN) images belong to different modalities with specific advantageous properties. Therefore, there is a large representation gap between them. Moreover, the features extracted independently by the two branches belong to different feature spaces, which is not conducive to the subsequent collaborative classification. At the same time, different layers also have different representation capabilities for objects with large size differences. In order to dynamically and adaptively transfer the dominant attributes, reduce the gap between them, find the best shared layer representation, and fuse the features of different representation capabilities, this article proposes an adaptive migration collaborative network (AMC-Net) for multimodal remote-sensing (RS) images classification. First, for the input of the network, we combine principal component analysis (PCA) and nonsubsampled contourlet transformation (NSCT) to migrate the advantageous attributes of the PAN and the MS images to each other. This not only improves the quality of images themselves, but also increases the similarity between the two images, thereby reducing the representational gap between them and the pressure on the subsequent classification network. Second, for the interaction on the feature migrate branch, we design a feature progressive migration fusion unit (FPMF-Unit) based on the adaptive cross-stitch unit of correlation coefficient analysis (CCA), which can make the network automatically learn the features that need to be shared and migrated, aiming to find the best shared-layer representation for multifeature learning. And we design an adaptive layer fusion mechanism module (ALFM-Module), which can adaptively fuse features of different layers, aiming to clearly model the dependencies among multiple layers for different sized objects. Finally, for the output of the network, we add the calculation of the correlation coefficient to the loss function, which can make the network converge to the global optimum as much as possible. The experimental results indicate that AMC-Net can achieve competitive performance. And the code for the network framework is available at: https://github.com/ru-willow/A-AFM-ResNet.

SAMCNet for Spatial-configuration-based Classification: A Summary of Results

Integrating Spatial Prior Adapter for Enhancing SAM Performance in Medical Image Segmentation

Towards Spatially-Lucid AI Classification in Non-Euclidean Space: An Application for MxIF Oncology Data

MM-DCNet: A Multi-Scale and Multi-MOdality Dynamic Convolutional Network for Lung Cancer Subtypes Classification

A Spatial-Channel Collaborative Attention Network for Enhancement of Multiresolution Classification

MSCCNet: Multi-Scale Convolution-Capsule Network for Cervical Cell Classification

Multi-Class Cell Detection Using Spatial Context Representation

A Stereo Spatial Decoupling Network for Medical Image Classification.

Channel-spatial attention network for fewshot classification

Relationship Learning From Multisource Images via Spatial-Spectral Perception Network

Advancing multi-categorization and segmentation in brain tumors using novel efficient deep learning approaches

Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation

A Lightweight Spatially-Aware Classification Model for Breast Cancer Pathology Images

An Adaptive Migration Collaborative Network for Multimodal Image Classification.

Z-SSMNet: Zonal-aware Self-supervised Mesh Network for Prostate Cancer Detection and Diagnosis with Bi-parametric MRI

SAMA: A Self-and-Mutual Attention Network for Accurate Recurrence Prediction of Non-Small Cell Lung Cancer Using Genetic and CT Data

Spatial-MGCN: a novel multi-view graph convolutional network for identifying spatial domains with attention mechanism

Channel Spatial Collaborative Attention Network for Fine-Grained Classification of Cervical Cells.

3MT-Net: A Multi-modal Multi-task Model for Breast Cancer and Pathological Subtype Classification Based on a Multicenter Study

CSA-Net: Channel and Spatial Attention-Based Network for Mammogram and Ultrasound Image Classification

SCSGNet: Spatial-Correlated and Shape-Guided Network for Breast Mass Segmentation