Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel Principles

Wei Cui,Zhanyun Feng,Jiale Chen,Xing Xu,Yueling Tian,Huilin Zhao,Chenglei Wang,Cui,Feng,Chen,Xu,Tian,Zhao,Wang

DOI: https://doi.org/10.3390/rs16081398

IF: 5

2024-04-16

Remote Sensing

Abstract:The performance of semantic segmentation in remote sensing, based on deep learning models, depends on the training data. A commonly encountered issue is the imbalanced long-tailed distribution of data, where the head classes contain the majority of samples while the tail classes have fewer samples. When training with long-tailed data, the head classes dominate the training process, resulting in poorer performance in the tail classes. To address this issue, various strategies have been proposed, such as resampling, reweighting, and transfer learning. However, common resampling methods suffer from overfitting to the tail classes while underfitting the head classes, and reweighting methods are limited in the extreme imbalanced case. Additionally, transfer learning tends to transfer patterns learned from the head classes to the tail classes without rigorously validating its generalizability. These methods often lack additional information to assist in the recognition of tail class objects, thus limiting performance improvements and constraining generalization ability. To tackle the abovementioned issues, a graph neural network based on the graph kernel principle is proposed for the first time. By leveraging the graph kernel, structural information for tail class objects is obtained, serving as additional contextual information beyond basic visual features. This method partially compensates for the imbalance between tail and head class object information without compromising the recognition accuracy of head classes objects. The experimental results demonstrate that this study effectively addresses the poor recognition performance of small and rare targets, partially alleviates the issue of spectral confusion, and enhances the model's generalization ability.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary

What problem does this paper attempt to address?

The paper attempts to address the issue of poor recognition performance of tail classes (i.e., categories with fewer samples) in remote sensing semantic segmentation tasks due to the long-tail effect of data distribution. Specifically, the paper points out that in deep learning-based remote sensing semantic segmentation, the imbalanced distribution of training data is a common problem, where the number of samples in head classes is large, while the number of samples in tail classes is small. This imbalance can cause the model to overly focus on head classes during the training process, thereby affecting the recognition performance of tail classes.

To tackle this challenge, the paper proposes a graph neural network method based on the principle of graph kernels. This method enhances the feature representation of tail classes by extracting the graph structure information of tail class objects and their neighboring nodes as supplementary information, thereby partially compensating for the information imbalance between head and tail classes. Experimental results show that this method effectively improves the recognition performance of small and rare targets, partially alleviates the spectral confusion problem, and enhances the generalization ability of the model.