Abstract:The effective combination of hyperspectral image (HSI) and light detection and ranging (LiDAR) data can be used for land cover classification. Recently, deep-learning-based classification methods, especially those using transformer networks, have achieved remarkable success. However, deep learning classification methods for multisource data still encounter various technical challenges, such as comprehensive utilization of multiscale information, lightweight network design, and efficient fusion strategies for heterogeneous data. To address these challenges, we propose a novel and efficient deep neural network, namely, multiscale 3-D-2-D mixed CNN feature extraction and multisource data lightweight attention-free fusion network (M2FNet) based on CNN and transformer. Through end-to-end training, this network effectively combines heterogeneous information from multiple sources, leading to improved performance in joint classification. Specifically, M2FNet uses a multiscale 3-D-2-D mixed CNN design to extract both the spatial-spectral features of HSI and the depth-based elevation features of LiDAR data. Subsequently, the extracted features are fed into a novel encoder comprising a feature enhancement (FE) module, designed with mathematical morphology and a dilated convolutional module derived from the self-attention of the conventional transformer encoder (DConvformer), which plays a crucial role in integrating multisource information within the network. The well-designed architecture enables the network to acquire multiscale depth and high-order features, significantly reducing the number of training parameters. Comparative experimental results and ablation studies demonstrate that M2FNet outperforms other advanced methods.

Hyperbolic Capsule Networks For Multi-Label Classification

Enhancing Deep Learning-Based Multi-label Text Classification with Capsule Network

A Capsule Network for Hierarchical Multi-Label Image Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Multi-label classification of legal text based on label embedding and capsule network

Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification

Hyperspectral image classification using multi-level features fusion capsule network with a dense structure

Multi-Scale Depthwise Separable Capsule Network for hyperspectral image classification

Hyperbolic Interaction Model For Hierarchical Multi-Label Classification

A Context-aware Capsule Network for Multi-label Classification

Hierarchical Multilabel Ship Classification in Remote Sensing Images Using Label Relation Graphs

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Multi-Task Learning with Capsule Networks.

A novel capsule network based on deep routing and residual learning

Deep multi-prototype capsule networks

Galaxy Morphology Classification Using Multi-Scale Convolution Capsule Network

Hybrid Gromov-Wasserstein Embedding for Capsule Learning

Complex Object Classification

Capsule Network Performance on Complex Data

Spectral–Spatial Hyperspectral Image Classification Using Dual-Channel Capsule Networks

1D-Convolutional Capsule Network for Hyperspectral Image Classification