Abstract:Mixup-based data augmentation has been proven to be beneficial to the regularization of models during training, especially in the remote-sensing field where the training data is scarce. However, in the process of data augmentation, the Mixup-based methods ignore the target proportion in different inputs and keep the linear insertion ratio consistent, which leads to the response of label space even if no effective objects are introduced in the mixed image due to the randomness of the augmentation process. Moreover, although some previous works have attempted to utilize different multimodal interaction strategies, they could not be well extended to various remote-sensing data combinations. To this end, a multistage information complementary fusion network based on flexible-mixup (Flex-MCFNet) is proposed for hyperspectral-X image classification. First, to bridge the gap between the mixed image and the label, a flexible-mixup (FlexMix) data augmentation strategy is designed, where the weight of the label increases with the ratio of the input image to prevent the negative impact on the label space because of the introduction of invalid information. More importantly, to summarize diverse remote-sensing data inputs including various modal supplements and uncertainties, a multistage information complementary fusion network (MCFNet) is developed. After extracting the features of hyperspectral and complementary modalities X-modal, including multispectral, synthetic aperture radar (SAR), and light detection and ranging (LiDAR) separately, the information between complementary modalities is fully interacted and enhanced through multiple stages of information complement and fusion, which is used for the final image classification. Extensive experimental results have demonstrated that Flex-MCFNet can not only effectively expand the training data, but also adequately regularize different data combinations to achieve state-of-the-art performance.

Multisource Joint Representation Learning Fusion Classification for Remote Sensing Images.

Hyperspectral and SAR Image Classification via Multiscale Interactive Fusion Network

Multisource Remote Sensing Data Classification with Graph Fusion Network

Multisource Image Classification Method Based on Information Fusion in Remote Sensing

Classification of Multisource Remote Sensing Images Using Multimodal Equilateral Absorption Network

Multiple Information Collaborative Fusion Network for Joint Classification of Hyperspectral and LiDAR Data

A Spatial-Spectral Relation-Guided Fusion Network for Multisource Optical RS Image Classification

MISP-Fuse: A progressive fusion network guided by Multi-Information supervision

MGMNet: Mutual-Guidance Mechanism for Joint Classification of Multisource Remote Sensing Data

Information Fusion for Classification of Hyperspectral and LiDAR Data Using IP-CNN

Multicrop Fusion Strategy Based on Prototype Assignment for Remote Sensing Image Scene Classification.

Target recognition with fusion of visible and infrared images based on mutual learning

A Joint Convolutional Cross ViT Network for Hyperspectral and Light Detection and Ranging Fusion Classification

Deep Multi-Level Fusion Network for Multi-Source Image Pixel-Wise Classification

A Spatial-Spectral Bilinear Representation Fusion Network for Multimodal Classification

Intra- and Intersource Interactive Representation Learning Network for Remote Sensing Images Classification

Intra- and Inter-source Interactive Representation Learning Network for Remote Sensing Images Classification

A Multistage Information Complementary Fusion Network Based on Flexible-Mixup for HSI-X Image Classification

Multiview Feature Learning and Multilevel Information Fusion for Joint Classification of Hyperspectral and LiDAR Data

A Novel MRF-Based Multifeature Fusion for Classification of Remote Sensing Images

Multimodal Hyperspectral Image Classification via Interconnected Fusion