Abstract:Hyperspectral images (HSIs) contain spatially structured information and pixel−level sequential spectral attributes. The continuous spectral features contain hundreds of wavelength bands and the differences between spectra are essential for achieving fine−grained classification. Due to the limited receptive field of backbone networks, convolutional neural networks (CNNs)−based HSI classification methods show limitations in modeling spectral−wise long−range dependencies with fixed kernel size and a limited number of layers. Recently, the self−attention mechanism of transformer framework is introduced to compensate for the limitations of CNNs and to mine the long−term dependencies of spectral signatures. Therefore, many joint CNN and Transformer architectures for HSI classification have been proposed to obtain the merits of both networks. However, these architectures make it difficult to capture spatial–spectral correlation and CNNs distort the continuous nature of the spectral signature because of the over−focus on spatial information, which means that the transformer can easily encounter bottlenecks in modeling spectral−wise similarity and long−range dependencies. To address this problem, we propose a neighborhood enhancement hybrid transformer (NEHT) network. In particular, a simple 2D convolution module is adopted to achieve dimensionality reduction while minimizing the distortion of the original spectral distribution by stacked CNNs. Then, we extract group−wise spatial–spectral features in a parallel design to enhance the representation capability of each token. Furthermore, a feature fusion strategy is introduced to increase subtle discrepancies of spectra. Finally, the self−attention of transformer is employed to mine the long−term dependencies between the enhanced feature sequences. Extensive experiments are performed on three well−known datasets and the proposed NEHT network shows superiority over state−of−the−art (SOTA) methods. Specifically, our proposed method outperforms the SOTA method by 0.46%, 1.05% and 0.75% on average in overall accuracy, average accuracy and kappa coefficient metrics.

Adaptively heterogeneous transfer learning for hyperspectral image classification

Heterogeneous Transfer Learning for Hyperspectral Image Classification Based on Convolutional Neural Network

Heterogeneous Spectral-Spatial Feature Transfer With Structure Preserved Distribution Alignment for Hyperspectral Image Classification

Transferring CNN Ensemble for Hyperspectral Image Classification

Sensor-specific Transfer Learning for Hyperspectral Image Processing

Hyperspectral Image Classification Using Hierarchical Spatial-Spectral Transformer

Cross-Domain Hyperspectral Image Classification Based on Graph Convolutional Networks

Joint spectral-spatial hyperspectral classification based on transfer learning (SSTL) from red-green-blue (RGB) images

Learning Transferable Discriminative Knowledge from Attribute-Aligned Hyperspectral Images

Cross-Scene Deep Transfer Learning With Spectral Feature Adaptation for Hyperspectral Image Classification

Adaptive Hash Attention and Lower Triangular Network for Hyperspectral Image Classification

Active-Learning-Incorporated Deep Transfer Learning for Hyperspectral Image Classification

Hybrid 2D–3D convolution and pre-activated residual networks for hyperspectral image classification

Hybrid Spectral-Spatial Convolutional Network and Transformer with Mixup Regularization for Hyperspectral Image Classification

Transfer Learning-Based Hyperspectral Image Classification Using Residual Dense Connection Networks

Aggregated-Attention Transformation Network for Hyperspectral Image Classification

Convolutional Neural Networks Based Hyperspectral Image Classification Method with Adaptive Kernels

Hyperspectral Image Classification via Spectral Pooling and Hybrid Transformer

Soft Instance-Level Domain Adaptation With Virtual Classifier for Unsupervised Hyperspectral Image Classification

Learning and Transferring Deep Joint Spectral–Spatial Features for Hyperspectral Classification