Abstract:It is a crucial means for humans to perceive geomorphic features and landscape architectures by classifying ground objects in hyperspectral images (HSIs). Currently, the exponential development of neural networks has provided a powerful support for the accurate HSI classification. However, existing neural network-based methods usually rely solely on the data to drive the classification model, lacking attention to valuable land-cover distribution knowledge in HSIs. In view of this, to utilize hyperspectral data and distribution knowledge simultaneously, a data and knowledge-driven deep multiview fusion network based on diffusion model (DKDMN) is proposed in this paper. DKDMN extracts knowledge from unlabeled data in HSIs through a diffusion model-based knowledge learning framework (DMKLF), and combines raw hyperspectral data with the acquired knowledge through a designed deep multiview network architecture (DMNA) to mine complicated land-cover distribution information and reflect sample relationships. First, the proposed DMKLF utilizes the data distribution reconstructed by the diffusion model as a knowledge source for one view to enhance the network cross-sample awareness ability. On the other hand, the original HSI patches are considered a data source for another view, which co-drives DMNA with the unsupervised diffusion knowledge extracted by DMKLF to perform effective feature extraction. Second, taking into account the characteristics of each view and the feature similarity between these two views, a joint loss function specifically for DMNA is suggested to minimize the difference between the model predictions and the real labels. Finally, a multi-backbone integration classification framework (MBICF) is designed by deeply fusing three vision architectures to capture multi-scale spectral features and local–global features, thereby achieving pixel-wise classification effectively. Experimental results on four publicly available HSI datasets demonstrate that the proposed DKDMN achieves competitive classification accuracy compared with other state-of-the-art methods. For instance, the proposed DKDMN achieves an overall accuracy improvement of 1.62% and 2.18% on the Indian Pines and Salinas Valley datasets, respectively, compared to the multiple vision architecture-based hybrid network (MVAHN). The related code will be released at https://github.com/ZJier/DKDMN .

Multidimensional Information Expansion and Processing Network for Hyperspectral Image Classification

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Dilated Deep MPFormer Network for Hyperspectral Image Classification

Multi-Level Feature Extraction Networks for Hyperspectral Image Classification

Hyperspectral Image Classification Based on Multilevel Joint Feature Extraction Network

Multilevel Features Fused and Change Information Enhanced Neural Network for Hyperspectral Image Change Detection

Three-dimensional Densely Connected Convolutional Network for Hyperspectral Remote Sensing Image Classification

Hyperspectral Image Classification Using Multi-Branch-multi-scale Residual Fusion Network

Hyperspectral Image Classification Based on Interactive Transformer and CNN With Multilevel Feature Fusion Network

AMHFN: Aggregation Multi-Hierarchical Feature Network for Hyperspectral Image Classification

Data and knowledge-driven deep multiview fusion network based on diffusion model for hyperspectral image classification

Lightweight Multilevel Feature Fusion Network for Hyperspectral Image Classification

Efficient Deep Learning of Nonlocal Features for Hyperspectral Image Classification

Lightweight Spatial-Spectral Network Based on 3D-2D Multi-Group Feature Extraction Module for Hyperspectral Image Classification

Efficient Deep Learning of Non-local Features for Hyperspectral Image Classification

Pseudo Complex-Valued Deformable ConvLSTM Neural Network With Mutual Attention Learning for Hyperspectral Image Classification

An Effective Hyperspectral Image Classification Network Based on Multi-Head Self-Attention and Spectral-Coordinate Attention

Hyperspectral Image Classification with a Dense Multi-Scale Residual Network

A Novel Classification Framework for Hyperspectral Image Data by Improved Multilayer Perceptron Combined with Residual Network

A Hybrid 3D–2D Feature Hierarchy CNN with Focal Loss for Hyperspectral Image Classification

A Feature Embedding Network with Multiscale Attention for Hyperspectral Image Classification