Semi-TSGAN: Semi-Supervised Learning for Highlight Removal Based on Teacher-Student Generative Adversarial Network

Yuanfeng Zheng,Yuchen Yan,Hao Jiang

DOI: https://doi.org/10.3390/s24103090

IF: 3.9

2024-05-14

Sensors

Abstract:Despite recent notable advancements in highlight image restoration techniques, the dearth of annotated data and the lightweight deployment of highlight removal networks pose significant impediments to further advancements in the field. In this paper, to the best of our knowledge, we first propose a semi-supervised learning paradigm for highlight removal, merging the fusion version of a teacher–student model and a generative adversarial network, featuring a lightweight network architecture. Initially, we establish a dependable repository to house optimal predictions as pseudo ground truth through empirical analyses guided by the most reliable No-Reference Image Quality Assessment (NR-IQA) method. This method serves to assess rigorously the quality of model predictions. Subsequently, addressing concerns regarding confirmation bias, we integrate contrastive regularization into the framework to curtail the risk of overfitting on inaccurate labels. Finally, we introduce a comprehensive feature aggregation module and an extensive attention mechanism within the generative network, considering a balance between network performance and computational efficiency. Our experimental evaluations encompass comprehensive assessments on both full-reference and non-reference highlight benchmarks. The results demonstrate conclusively the substantive quantitative and qualitative enhancements achieved by our proposed algorithm in comparison to state-of-the-art methodologies.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

The main issues that this paper attempts to address are the lack of data annotation in highlight removal techniques and the lightweight deployment of networks. Specifically:

- **Lack of Data Annotation**: Current highlight removal algorithms typically rely on supervised learning, requiring a large amount of high-quality paired datasets (i.e., pairs of images with and without highlights). However, collecting these paired datasets requires significant human, material, and financial resources, and poses many challenges in practical operations.
- **Lightweight Deployment**: Existing highlight removal models based on Generative Adversarial Networks (GANs) often have complex structures and a large number of parameters, making actual deployment difficult, especially in resource-constrained application scenarios.

To address these issues, the authors propose a new semi-supervised learning framework—Semi-TSGAN (Semi-Supervised Learning Method based on Teacher-Student Generative Adversarial Networks), which can fully utilize unlabeled data to enhance the adaptability and performance of the model. Additionally, this framework designs a lightweight network architecture to balance network performance and computational efficiency. By introducing contrastive loss and a reliable pseudo-label repository, this method aims to reduce the risk of overfitting and improve the generalization ability of the model. Experimental results show that this method achieves significant performance improvements in various benchmark tests.