Abstract:Hyperspectral images (HSIs) contain spatially structured information and pixel−level sequential spectral attributes. The continuous spectral features contain hundreds of wavelength bands and the differences between spectra are essential for achieving fine−grained classification. Due to the limited receptive field of backbone networks, convolutional neural networks (CNNs)−based HSI classification methods show limitations in modeling spectral−wise long−range dependencies with fixed kernel size and a limited number of layers. Recently, the self−attention mechanism of transformer framework is introduced to compensate for the limitations of CNNs and to mine the long−term dependencies of spectral signatures. Therefore, many joint CNN and Transformer architectures for HSI classification have been proposed to obtain the merits of both networks. However, these architectures make it difficult to capture spatial–spectral correlation and CNNs distort the continuous nature of the spectral signature because of the over−focus on spatial information, which means that the transformer can easily encounter bottlenecks in modeling spectral−wise similarity and long−range dependencies. To address this problem, we propose a neighborhood enhancement hybrid transformer (NEHT) network. In particular, a simple 2D convolution module is adopted to achieve dimensionality reduction while minimizing the distortion of the original spectral distribution by stacked CNNs. Then, we extract group−wise spatial–spectral features in a parallel design to enhance the representation capability of each token. Furthermore, a feature fusion strategy is introduced to increase subtle discrepancies of spectra. Finally, the self−attention of transformer is employed to mine the long−term dependencies between the enhanced feature sequences. Extensive experiments are performed on three well−known datasets and the proposed NEHT network shows superiority over state−of−the−art (SOTA) methods. Specifically, our proposed method outperforms the SOTA method by 0.46%, 1.05% and 0.75% on average in overall accuracy, average accuracy and kappa coefficient metrics.

Multisource Remote Sensing Image Classification Based on Adaptive Pooling Transformer and Multilevel Correction

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Fusing Deep Local And Global Features For Remote Sensing Image Scene Classification

TCPSNet: Transformer and Cross-Pseudo-Siamese Learning Network for Classification of Multi-Source Remote Sensing Images

Multisource Remote Sensing Data Classification Based on Convolutional Neural Network

A Novel Adaptive Hybrid Fusion Network for Multiresolution Remote Sensing Images Classification

Hierarchical Attention and Parallel Filter Fusion Network for Multisource Data Classification

Joint Classification of Hyperspectral and LiDAR Data Using a Hierarchical CNN and Transformer

Multi-source remote sensing image classification based on two-channel densely connected convolutional networks

Hierarchical Attention and Parallel Filter Fusion Network for Multi-Source Data Classification

A multimodal hyper-fusion transformer for remote sensing image classification

MPT-SFANet: Multi-Order Pooling Transformer-Based Semantic Feature Aggregation Network for SAR Image Classification

Hyperspectral and SAR Image Classification via Multiscale Interactive Fusion Network

A Dual-Branch Attention Fusion Deep Network for Multiresolution Remote-Sensing Image Classification

A Multi-Scale Progressive Collaborative Attention Network for Remote Sensing Fusion Classification.

A Joint Convolutional Cross ViT Network for Hyperspectral and Light Detection and Ranging Fusion Classification

High Resolution Remote Sensing Image Classification Combining with Mean-Shift Segmentation and Fully Convolution Neural Network

A Lightweight and Discriminative Model for Remote Sensing Scene Classification with Multidilation Pooling Module

Dual-Branch Feature Fusion Network Based Cross-Modal Enhanced CNN and Transformer for Hyperspectral and LiDAR Classification

Hyperspectral Image Classification via Spectral Pooling and Hybrid Transformer

Hybrid FusionNet: A Hybrid Feature Fusion Framework for Multi-Source High-Resolution Remote Sensing Image Classification