A Lightweight Convolution Network with Self-Knowledge Distillation for Hyperspectral Image Classification

Hao Xu,Guo Cao,Lindiao Deng,Lanwei Ding,Ling Xu,Qikun Pan,Yanfeng Shang
DOI: https://doi.org/10.1117/12.2680012
2022-01-01
Abstract:Recently, using convolutional neural networks (CNNs) to extract spectral-spatial features has become an effective way for HSI classification. However, complex CNN models require many training parameters and floating-point operations (FLOPs). This usually means longer training and testing times. Furthermore, deep networks become prone to overfitting when the labeled samples of hyperspectral data are limited. In this article, a lightweight convolution network with self-knowledge distillation (SKDLCN) is developed for HSI classification, and it has two crucial elements, including a dual-path convolution network and a self-knowledge distillation module. At first, a method called 3-D transformation is performed for data augmentation to alleviate the overfitting problem. Then, the proposed network consists of small 1 x1 convolutions with a residual path and a density path. Specifically, it can efficiently complete the extraction of spectral and spectral-spatial features sequentially from HSI. Self-knowledge distillation can be explained within the knowledge distillation framework as students become teachers, which gradually extracts knowledge of the model itself during training. Specifically, the target is adaptively adjusted by combining the ground truth of the model itself and past predictions. Experiments on two public HSI datasets demonstrate that the proposed method is significantly superior to some state-of-the-art methods with limited training samples.
What problem does this paper attempt to address?