Abstract:The effective combination of hyperspectral image (HSI) and light detection and ranging (LiDAR) data can be used for land cover classification. Recently, deep-learning-based classification methods, especially those using transformer networks, have achieved remarkable success. However, deep learning classification methods for multisource data still encounter various technical challenges, such as comprehensive utilization of multiscale information, lightweight network design, and efficient fusion strategies for heterogeneous data. To address these challenges, we propose a novel and efficient deep neural network, namely, multiscale 3-D-2-D mixed CNN feature extraction and multisource data lightweight attention-free fusion network (M2FNet) based on CNN and transformer. Through end-to-end training, this network effectively combines heterogeneous information from multiple sources, leading to improved performance in joint classification. Specifically, M2FNet uses a multiscale 3-D-2-D mixed CNN design to extract both the spatial-spectral features of HSI and the depth-based elevation features of LiDAR data. Subsequently, the extracted features are fed into a novel encoder comprising a feature enhancement (FE) module, designed with mathematical morphology and a dilated convolutional module derived from the self-attention of the conventional transformer encoder (DConvformer), which plays a crucial role in integrating multisource information within the network. The well-designed architecture enables the network to acquire multiscale depth and high-order features, significantly reducing the number of training parameters. Comparative experimental results and ablation studies demonstrate that M2FNet outperforms other advanced methods.

Lightweight Transformer with Transposed-attention for Hyperspectral Unmixing

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

A Novel Transformer Network with a CNN-Enhanced Cross-Attention Mechanism for Hyperspectral Image Classification

Spatial-spectral collaborative attention network for hyperspectral unmixing

An Effective Unmixing Network for Deep Spectral Information Extraction

Efficient Blind Hyperspectral Unmixing with Non-Local Spatial Information Based on Swin Transformer

Transformer for Multitemporal Hyperspectral Image Unmixing

A Lightweight Transformer Network for Hyperspectral Image Classification

Transformer based Endmember Fusion with Spatial Context for Hyperspectral Unmixing

UnDAT: Double-Aware Transformer for Hyperspectral Unmixing.

CTNet: an Efficient Coupled Transformer Network for Robust Hyperspectral Unmixing

Deep Hyperspectral Unmixing using Transformer Network

TCCU-Net: Transformer and CNN Collaborative Unmixing Network for Hyperspectral Image

Multiscale and Cross-Level Attention Learning for Hyperspectral Image Classification

MSCC-ViT:A Multiscale Visual-Transformer Network Using Convolution Crossing Attention for Hyperspectral Unmixing

Window Transformer Convolutional Autoencoder for Hyperspectral Sparse Unmixing

CDCTA: cascaded dual-constrained transformer autoencoder for hyperspectral unmixing with endmember variability and spectral geometry

ELS2T: Efficient Lightweight Spectral-Spatial Transformer for Hyperspectral Image Classification

Ultralightweight Feature-Compressed Multihead Self-Attention Learning Networks for Hyperspectral Image Classification

A Lightweight 1-D Convolution Augmented Transformer with Metric Learning for Hyperspectral Image Classification

SPFormer: Self-Pooling Transformer for Few-Shot Hyperspectral Image Classification