Hyperspectral Image Classification Using Feature Fusion Hypergraph Convolution Neural Network

Zhongtian Ma,Zhiguo Jiang,Haopeng Zhang

DOI: https://doi.org/10.1109/tgrs.2021.3123423

IF: 8.2

2022-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Convolution neural networks (CNNs) and graph representation learning are two common methods for hyperspectral image (HSI) classification. Recently, graph convolutional neural networks, a combination of CNN and graph representation learning, have shown great potential in the HSI classification problem. However, the existing graph convolution network (GCN)-based methods have many problems, such as overdependence on the adjacency matrix, usage of a single modal feature, and lower accuracy than the mature CNN method. In this article, we propose a feature fusion hypergraph neural network (F2HNN) for HSI classification. F2HNN first generates hyperedges from features of different modalities to construct a hypergraph representing multimodal features in HSI. Then, the HSI and the extracted hypergraph are input into the hypergraph convolutional neural network for learning. In addition, we propose three feature fusion strategies. The first strategy is the most basic spatial and spectral feature fusion. The second strategy fuses the spectral features extracted by a pretrained multilayer perceptron (MLP) with the spatial features to reduce the redundant information of the original spectral features. The third strategy uses the fusion of CNN features, spectral features, and spatial features to explore the capabilities of F2HNN. Sufficient experiments on four datasets have proved the effectiveness of F2HNN.