Abstract:It is a crucial means for humans to perceive geomorphic features and landscape architectures by classifying ground objects in hyperspectral images (HSIs). Currently, the exponential development of neural networks has provided a powerful support for the accurate HSI classification. However, existing neural network-based methods usually rely solely on the data to drive the classification model, lacking attention to valuable land-cover distribution knowledge in HSIs. In view of this, to utilize hyperspectral data and distribution knowledge simultaneously, a data and knowledge-driven deep multiview fusion network based on diffusion model (DKDMN) is proposed in this paper. DKDMN extracts knowledge from unlabeled data in HSIs through a diffusion model-based knowledge learning framework (DMKLF), and combines raw hyperspectral data with the acquired knowledge through a designed deep multiview network architecture (DMNA) to mine complicated land-cover distribution information and reflect sample relationships. First, the proposed DMKLF utilizes the data distribution reconstructed by the diffusion model as a knowledge source for one view to enhance the network cross-sample awareness ability. On the other hand, the original HSI patches are considered a data source for another view, which co-drives DMNA with the unsupervised diffusion knowledge extracted by DMKLF to perform effective feature extraction. Second, taking into account the characteristics of each view and the feature similarity between these two views, a joint loss function specifically for DMNA is suggested to minimize the difference between the model predictions and the real labels. Finally, a multi-backbone integration classification framework (MBICF) is designed by deeply fusing three vision architectures to capture multi-scale spectral features and local–global features, thereby achieving pixel-wise classification effectively. Experimental results on four publicly available HSI datasets demonstrate that the proposed DKDMN achieves competitive classification accuracy compared with other state-of-the-art methods. For instance, the proposed DKDMN achieves an overall accuracy improvement of 1.62% and 2.18% on the Indian Pines and Salinas Valley datasets, respectively, compared to the multiple vision architecture-based hybrid network (MVAHN). The related code will be released at https://github.com/ZJier/DKDMN .

DSNN: A Dynamic-Structure Neural Network for Aerial Target Multiview High-Resolution Range Profiles Classification.

Dynamic Convolution Covariance Network Using Multi-Scale Feature Fusion for Remote Sensing Scene Image Classification

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

Superpixel-Based Long-Range Dependent Network for High-Resolution Remote-Sensing Image Classification

Aerial Target Classification Using an LFL-Net with Multiview HRRPs

Deep Feature Fusion for High-Resolution Aerial Scene Classification

DenseNet-Based Depth-Width Double Reinforced Deep Learning Neural Network for High-Resolution Remote Sensing Image Per-Pixel Classification

Data and knowledge-driven deep multiview fusion network based on diffusion model for hyperspectral image classification

Multiview Deep Feature Learning Network for SAR Automatic Target Recognition

Multi-resolution Classification Network for High-Resolution UAV Remote Sensing Images

DSHFNet: Dynamic Scale Hierarchical Fusion Network Based on Multiattention for Hyperspectral Image and LiDAR Data Classification

Multiscale and Multidirection Feature Extraction Network for Hyperspectral and LiDAR Classification

SAR Automatic Target Recognition Based on Multiview Deep Learning Framework

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

Parallel multi-stage features fusion of deep convolutional neural networks for aerial scene classification

A Deep Neural Network Combined With Context Features for Remote Sensing Scene Classification

3-D Channel and Spatial Attention Based Multiscale Spatial-Spectral Residual Network for Hyperspectral Image Classification

Dynamic Convolution Covariance Network Using Multiscale Feature Fusion for Remote Sensing Scene Image Classification

Dual-branch Dense Residual Network for Hyperspectral Imagery Classification

Multiscale DenseNet Meets With Bi-RNN for Hyperspectral Image Classification

Attention-Mechanism-Containing Neural Networks for High-Resolution Remote Sensing Image Classification