Quality-guided Skin Tone Enhancement for Portrait Photography

Shiqi Gao,Huiyu Duan,Xinyue Li,Kang Fu,Yicong Peng,Qihang Xu,Yuanyuan Chang,Jia Wang,Xiongkuo Min,Guangtao Zhai
2024-06-22
Abstract:In recent years, learning-based color and tone enhancement methods for photos have become increasingly popular. However, most learning-based image enhancement methods just learn a mapping from one distribution to another based on one dataset, lacking the ability to adjust images continuously and controllably. It is important to enable the learning-based enhancement models to adjust an image continuously, since in many cases we may want to get a slighter or stronger enhancement effect rather than one fixed adjusted result. In this paper, we propose a quality-guided image enhancement paradigm that enables image enhancement models to learn the distribution of images with various quality ratings. By learning this distribution, image enhancement models can associate image features with their corresponding perceptual qualities, which can be used to adjust images continuously according to different quality scores. To validate the effectiveness of our proposed method, a subjective quality assessment experiment is first conducted, focusing on skin tone adjustment in portrait photography. Guided by the subjective quality ratings obtained from this experiment, our method can adjust the skin tone corresponding to different quality requirements. Furthermore, an experiment conducted on 10 natural raw images corroborates the effectiveness of our model in situations with fewer subjects and fewer shots, and also demonstrates its general applicability to natural images. Our project page is <a class="link-external link-https" href="https://github.com/IntMeGroup/quality-guided-enhancement" rel="external noopener nofollow">this https URL</a> .
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that the existing learning - based image enhancement methods lack the ability to continuously and controllably adjust images. Specifically, most learning - based image enhancement models are only based on one dataset to map from one distribution to another, and the output results are fixed, making it difficult to fine - tune according to different user requirements. Therefore, this paper proposes a quality - guided image enhancement framework, enabling the image enhancement model to continuously and controllably adjust images according to different quality scores. ### Main Problem Description 1. **Limitations of Existing Methods**: - Existing learning - type image enhancement methods are usually trained only based on one dataset, and the output results are fixed and cannot be flexibly adjusted according to user needs. - There is a lack of a method that can continuously adjust images according to different quality scores. 2. **Research Objectives**: - Propose a new quality - guided image enhancement framework, enabling the image enhancement model to continuously and controllably adjust images according to different quality scores. - Verify the effectiveness of this framework in tasks such as skin tone adjustment through experiments. ### Solution Overview To solve the above problems, the author proposes the following solutions: - **Quality - guided Image Enhancement Framework**: This framework enables the image enhancement model to learn the image distributions with different quality labels and associate image features with the corresponding perceptual quality scores. In this way, during the inference process, the model can adjust the image according to different quality scores. - **Application of 3D LUT Image Enhancement Method**: Due to the feasibility of 3D LUT in practical applications, the author first applies it to the 3D LUT image enhancement method. This framework guides the 3D LUT image enhancement model to learn the image distributions with different quality labels and associates image features with their corresponding perceptual quality scores. - **Skin Tone Image Quality Assessment Database (STIQAD)**: To verify the effectiveness of this framework, the author constructs a skin tone image quality assessment database (STIQAD), which includes 85 original portrait images and 1,105 adjusted facial images, and collects the Mean Opinion Scores (MOSs) from 20 subjects. - **Experimental Verification**: Through a series of experiments, including subjective quality assessment experiments and experiments on natural images, the effectiveness and generalization ability of this framework are verified. ### Conclusion The quality - guided image enhancement framework proposed in this paper can continuously and controllably adjust images according to different quality scores, solving the problem that existing methods are difficult to flexibly adjust according to user needs. The experimental results show that this framework has good performance and adaptability in tasks such as skin tone adjustment.