Dual-stage Hyperspectral Image Classification Model with Spectral Supertoken

Peifu Liu,Tingfa Xu,Jie Wang,Huan Chen,Huiyan Bai,Jianan Li

2024-07-13

Abstract:Hyperspectral image classification, a task that assigns pre-defined classes to each pixel in a hyperspectral image of remote sensing scenes, often faces challenges due to the neglect of correlations between spectrally similar pixels. This oversight can lead to inaccurate edge definitions and difficulties in managing minor spectral variations in contiguous areas. To address these issues, we introduce the novel Dual-stage Spectral Supertoken Classifier (DSTC), inspired by superpixel concepts. DSTC employs spectrum-derivative-based pixel clustering to group pixels with similar spectral characteristics into spectral supertokens. By projecting the classification of these tokens onto the image space, we achieve pixel-level results that maintain regional classification consistency and precise boundary. Moreover, recognizing the diversity within tokens, we propose a class-proportion-based soft label. This label adaptively assigns weights to different categories based on their prevalence, effectively managing data distribution imbalances and enhancing classification performance. Comprehensive experiments on WHU-OHS, IP, KSC, and UP datasets corroborate the robust classification capabilities of DSTC and the effectiveness of its individual components. Code will be publicly available at <a class="link-external link-https" href="https://github.com/laprf/DSTC" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily addresses the challenges in Hyperspectral Image Classification (HSIC), particularly how to handle the spectral similarity between adjacent pixels and the resulting issues of unclear boundary definitions and poor internal consistency within regions. The authors propose a new method called the Dual-stage Spectral Supertoken Classifier (DSTC) to tackle these problems.

### Main Issues

The paper aims to solve the following two main issues:
1. **Boundary Blur**: Existing methods often lack precision in handling the boundaries between different category regions, leading to blurred boundaries in the classification results.
2. **Poor Internal Consistency**: For continuous regions with small spectral variations, existing methods may produce inconsistent classification results.

### Solution

To address the above issues, the authors propose DSTC, a dual-stage classifier. This method first clusters spectrally similar pixels using spectral derivative features to form "Spectral Supertokens," then classifies these supertokens and maps the classification results back to the original image space to obtain the final pixel-level classification results. Specifically:

1. **Stage 1: Spectral Supertoken Generation**
- Use spectral derivative-based pixel clustering techniques to pre-cluster spectrally similar pixels.
- Calculate the association degree of each pixel with the cluster centers and group them into the corresponding clusters.
- Form spectral supertokens based on the clustering results.

2. **Stage 2: Token-to-Pixel Prediction**
- Use a Transformer model to classify the spectral supertokens.
- Project the classified supertoken results back to the image space to obtain pixel-level classification results.

Additionally, to better handle the issue of uneven data distribution, the paper introduces a Class-Proportion-Based Soft Label (CPSL). This labeling strategy considers the proportion of different categories within each supertoken, effectively mitigating the impact of data imbalance.

### Experimental Results

The paper conducts experimental validation on multiple hyperspectral image datasets such as WHU-OHS, IP, KSC, and UP. The experimental results show that DSTC outperforms other comparison methods on these datasets, achieving significant improvements in classification accuracy. Particularly when dealing with large-scale and high-resolution datasets, DSTC demonstrates strong classification performance.

In summary, DSTC effectively addresses the key issues in hyperspectral image classification by introducing a dual-stage processing flow and a class-proportion-based soft label mechanism, improving classification accuracy and consistency.