Zheng-Peng Duan,Jiawei zhang,Zheng Lin,Xin Jin,Dongqing Zou,Chunle Guo,Chongyi Li
Abstract:Image retouching aims to enhance the visual quality of photos. Considering the different aesthetic preferences of users, the target of retouching is subjective. However, current retouching methods mostly adopt deterministic models, which not only neglects the style diversity in the expert-retouched results and tends to learn an average style during training, but also lacks sample diversity during inference. In this paper, we propose a diffusion-based method, named DiffRetouch. Thanks to the excellent distribution modeling ability of diffusion, our method can capture the complex fine-retouched distribution covering various visual-pleasing styles in the training data. Moreover, four image attributes are made adjustable to provide a user-friendly editing mechanism. By adjusting these attributes in specified ranges, users are allowed to customize preferred styles within the learned fine-retouched distribution. Additionally, the affine bilateral grid and contrastive learning scheme are introduced to handle the problem of texture distortion and control insensitivity respectively. Extensive experiments have demonstrated the superior performance of our method on visually appealing and sample diversity. The code will be made available to the community.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper aims to solve several key problems in image retouching:
1. **Lack of style diversity**: Most of the current retouching methods use deterministic models. These models ignore the style diversity in the expert retouching results during the training process and often learn an average style. This leads to a lack of diversity and personalization in the generated retouching results.
2. **Single style in the inference process**: Existing retouching methods can only generate one retouching style during the inference process, which may not meet users' aesthetic preferences. In order to generate results in multiple styles, multiple models need to be trained, which limits the flexibility of practical applications.
3. **Requirement for additional examples**: Although some methods support generating additional styles, they usually require users to provide additional examples to indicate the desired style, which increases the user's burden, and the way of adjusting the style through the image is not clear.
4. **Texture distortion and insensitive control**: Existing retouching methods have information loss during the encoding and decoding processes, resulting in texture distortion in the generated images. In addition, the impact of adjusting image properties is usually weak and cannot meet the actual needs.
### Solutions
To solve the above problems, the author proposes a retouching method based on the diffusion model, called **DiffRetouch**. The main features and innovations of this method include:
1. **Capturing complex distributions**: By using the excellent distribution modeling ability of the diffusion model, DiffRetouch can capture the complex distributions of various visually pleasing styles in the training data, even if the training data contains the retouching results of multiple experts.
2. **Adjustable image properties**: Four adjustable image properties (color richness, contrast, color temperature, and brightness) are introduced and adjusted by coefficients, providing a user - friendly editing mechanism. Users can adjust these properties within a specified range to customize their favorite styles.
3. **Affine bilateral grid**: To solve the texture distortion problem, an affine bilateral grid is introduced. By outputting the affine bilateral grid in each denoising step and directly applying it to the input image in the last step, texture distortion can be effectively reduced.
4. **Contrastive learning scheme**: To solve the insensitive control problem, a contrastive learning scheme is designed. Through explicit supervision of each property, the model can respond more sensitively to the adjustment of each coefficient.
### Experimental results
The experimental results show that DiffRetouch performs well in terms of visual attractiveness and sample diversity. Specifically:
- **Quantitative evaluation**: The experimental results on the MIT - Adobe FiveK and PPR10K datasets show that DiffRetouch outperforms existing methods in terms of PSNR, SSIM, LPIPS and other indicators.
- **Qualitative evaluation**: The generated retouching results are not only visually pleasing but also can cover various styles, meeting the aesthetic preferences of different users.
In conclusion, DiffRetouch successfully solves multiple problems existing in the current retouching methods by introducing the diffusion model and a series of innovative technologies, providing users with a flexible and high - quality automatic retouching tool.