Abstract:In multimodal data fusion and land-cover interpretation tasks, the fusion interpretability between hyperspectral image (HSI) and light detection and ranging (LiDAR) data is always nontrivial to be clarified. Furthermore, the heterogeneous sample and distribution variances of these two remote sensing (RS) modalities impede the joint classification performance. In this article, a hyperspectral intrinsic image decomposition guided data fusion network (HI(2)D(2)FNet) is proposed. Generally, classic hyperspectral intrinsic image decomposition (HIID) performs well in image enhancement and shadow removal. It decomposes one HSI into one reflectance component and one shading component. Inspired by the core mechanism of HIID, our motivation is to preliminarily exploit its potential for multimodal RS data fusion and explore the inherent modality connection between HSI and LiDAR data from the intrinsic perspective. Specifically, compared with existing techniques, HI(2)D(2)FNet is capable of fusing the horizontal geometry information in the shading component with the vertical geometry information in the LiDAR data from both sample and distribution perspectives in the spatial domain. The generated cross-modal geometry feature contributes to guiding the reflectance stream optimization. This unique fusion framework connects both the modalities with respect to geometry information and enhances the specific fusion interpretability. The decomposition and fusion modules in HI(2)D(2)FNet are optimized simultaneously in a novel alternative optimization pattern. Furthermore, several unique cross-modal constraints in terms of prior RS properties are presented. Experiments conducted on three widely available datasets prove the superiority of HI(2)D(2)FNet over state-of-the-art techniques. The source codes will be available at https://github.com/GEOywb/HI2D2FNet.

PID-HLfusion: Pluggable Progressive Illumination Driven Hyperspectral and LiDAR Data Fusion Considering Crossmodal Geometric Structures

HI2D2FNet: Hyperspectral Intrinsic Image Decomposition Guided Data Fusion Network for Hyperspectral and LiDAR Classification

Shadow Mask-Driven Multimodal Intrinsic Image Decomposition for Hyperspectral and LiDAR Data Fusion.

Multimodal Hyperspectral Image Classification via Interconnected Fusion

LiDAR-Guided Cross-Attention Fusion for Hyperspectral Band Selection and Image Classification

A Hierarchical Coarse–Fine Adaptive Fusion Network for the Joint Classification of Hyperspectral and LiDAR Data

Fusion of Hyperspectral-Multispectral images joining Spatial-Spectral Dual-Dictionary and structured sparse Low-rank representation

Unified Cloud-Based Framework for Hyperspectral and Multispectral Image Fusion Incorporating Nonlocal Principles and Tensor Decomposition

PIAFusion: A progressive infrared and visible image fusion network based on illumination aware

Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model

Physics-Inspired Degradation Models for Hyperspectral Image Fusion

Feature-Decision Level Collaborative Fusion Network for Hyperspectral and LiDAR Classification

Integrated Fusion for Panchromatic, Multispectral, Hyperspectral Remote Sensing Images With Different Swath Widths

Deep-Learning-Based Spatio-Temporal-Spectral Integrated Fusion of Heterogeneous Remote Sensing Images

Joint Classification of Hyperspectral and LiDAR Data Using Height Information Guided Hierarchical Fusion-and-Separation Network

Dual-Branch Feature Fusion Network Based Cross-Modal Enhanced CNN and Transformer for Hyperspectral and LiDAR Classification

Information Fusion for Classification of Hyperspectral and LiDAR Data Using IP-CNN

CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation

Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image Fusion

An Integrated Framework for the Heterogeneous Spatio-Spectral-Temporal Fusion of Remote Sensing Images