A Multitask Deep Learning Model for Classification and Regression of Hyperspectral Images: Application to the large-scale dataset

Koushikey Chhapariya,Alexandre Benoit,Krishna Mohan Buddhiraju,Anil Kumar

2024-07-23

Abstract:Multitask learning is a widely recognized technique in the field of computer vision and deep learning domain. However, it is still a research question in remote sensing, particularly for hyperspectral imaging. Moreover, most of the research in the remote sensing domain focuses on small and single-task-based annotated datasets, which limits the generalizability and scalability of the developed models to more diverse and complex real-world scenarios. Thus, in this study, we propose a multitask deep learning model designed to perform multiple classification and regression tasks simultaneously on hyperspectral images. We validated our approach on a large hyperspectral dataset called TAIGA, which contains 13 forest variables, including three categorical variables and ten continuous variables with different biophysical parameters. We design a sharing encoder and task-specific decoder network to streamline feature learning while allowing each task-specific decoder to focus on the unique aspects of its respective task. Additionally, a dense atrous pyramid pooling layer and attention network were integrated to extract multi-scale contextual information and enable selective information processing by prioritizing task-specific features. Further, we computed multitask loss and optimized its parameters for the proposed framework to improve the model performance and efficiency across diverse tasks. A comprehensive qualitative and quantitative analysis of the results shows that the proposed method significantly outperforms other state-of-the-art methods. We trained our model across 10 seeds/trials to ensure robustness. Our proposed model demonstrates higher mean performance while maintaining lower or equivalent variability. To make the work reproducible, the codes will be available at <a class="link-external link-https" href="https://github.com/Koushikey4596/Multitask-Deep-Learning-Model-for-Taiga-datatset" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the following issues:

1. **Development of a Multi-task Learning Framework**: The paper aims to develop a multi-task learning framework capable of simultaneously predicting different categories and computing continuous variable values, especially for large hyperspectral datasets. Currently, most studies focus on small and single-task annotated datasets, which limits the generalization and scalability of models in more diverse and complex real-world scenarios.

2. **Multi-task Processing of Hyperspectral Images**: The paper explores the behavior of multi-task learning on hyperspectral images, particularly in the presence of a large number of bands, and the impact of integrating spectral and spatial information on the performance of multi-task models.

3. **Data Imbalance and Task Balancing**: The paper proposes effective methods to address the issue of class imbalance in the data and achieve task balancing to compute multi-task loss and optimize its parameters, thereby improving the overall performance and efficiency of the model.

Specifically, the main contributions of the paper include:
- Developing a multi-task deep learning model capable of simultaneously predicting multiple categories and computing continuous variable values in large datasets.
- Designing a framework to analyze the behavior of multi-task methods on hyperspectral images, particularly considering the presence of a large number of bands, and evaluating the impact of integrating spectral and spatial information on the performance of multi-task models.
- Introducing cost-sensitive learning and focal loss methods to address the issue of class imbalance in the data and achieve task balancing, compute multi-task loss, and optimize its parameters, thereby enhancing the performance and efficiency of the model in different tasks.

Through these methods, the paper aims to improve the performance of multi-task learning in hyperspectral image classification and regression tasks while reducing the consumption of computational resources.