Graph Convolutional Networks for Hyperspectral Image Classification

Danfeng Hong,Lianru Gao,Jing Yao,Bing Zhang,Antonio Plaza,Jocelyn Chanussot

DOI: https://doi.org/10.1109/TGRS.2020.3015157

2021-01-15

Abstract:To read the final version please go to IEEE TGRS on IEEE Xplore. Convolutional neural networks (CNNs) have been attracting increasing attention in hyperspectral (HS) image classification, owing to their ability to capture spatial-spectral feature representations. Nevertheless, their ability in modeling relations between samples remains limited. Beyond the limitations of grid sampling, graph convolutional networks (GCNs) have been recently proposed and successfully applied in irregular (or non-grid) data representation and analysis. In this paper, we thoroughly investigate CNNs and GCNs (qualitatively and quantitatively) in terms of HS image classification. Due to the construction of the adjacency matrix on all the data, traditional GCNs usually suffer from a huge computational cost, particularly in large-scale remote sensing (RS) problems. To this end, we develop a new mini-batch GCN (called miniGCN hereinafter) which allows to train large-scale GCNs in a mini-batch fashion. More significantly, our miniGCN is capable of inferring out-of-sample data without re-training networks and improving classification performance. Furthermore, as CNNs and GCNs can extract different types of HS features, an intuitive solution to break the performance bottleneck of a single model is to fuse them. Since miniGCNs can perform batch-wise network training (enabling the combination of CNNs and GCNs) we explore three fusion strategies: additive fusion, element-wise multiplicative fusion, and concatenation fusion to measure the obtained performance gain. Extensive experiments, conducted on three HS datasets, demonstrate the advantages of miniGCNs over GCNs and the superiority of the tested fusion strategies with regards to the single CNN or GCN models. The codes of this work will be available at <a class="link-external link-https" href="https://github.com/danfenghong/IEEE_TGRS_GCN" rel="external noopener nofollow">this https URL</a> for the sake of reproducibility.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the following issues:

1. **Limited sample relationship modeling capability in hyperspectral image classification**: Although Convolutional Neural Networks (CNNs) perform well in hyperspectral image classification by capturing spatial-spectral feature representations, their ability to model relationships between samples is still limited.

2. **High computational cost of Graph Convolutional Networks (GCNs) on large-scale data**: Traditional GCNs face significant computational costs when dealing with large-scale remote sensing problems due to the need to construct the adjacency matrix for the entire dataset.

3. **GCNs only support full-batch training**: Traditional GCNs can only train the network in a full-batch manner, which may lead to high memory consumption, slow gradient descent, and negative impacts on variable updates.

4. **GCNs cannot directly predict new samples**: Trained GCN-based models cannot predict new input samples without retraining, which is a major limitation in practical applications.

To overcome these difficulties, the paper introduces a simple yet effective mini-batch Graph Convolutional Network (miniGCNs) and systematically analyzes the advantages and disadvantages of CNNs and GCNs in hyperspectral image classification tasks. Additionally, the paper proposes three fusion strategies (additive fusion, element-wise multiplicative fusion, and concatenation fusion) aimed at improving classification performance by integrating features extracted from CNNs and miniGCNs.