UIEDP:Underwater Image Enhancement with Diffusion Prior

Dazhao Du,Enhan Li,Lingyu Si,Fanjiang Xu,Jianwei Niu,Fuchun Sun

DOI: https://doi.org/10.48550/arXiv.2312.06240

2023-12-11

Abstract:Underwater image enhancement (UIE) aims to generate clear images from low-quality underwater images. Due to the unavailability of clear reference images, researchers often synthesize them to construct paired datasets for training deep models. However, these synthesized images may sometimes lack quality, adversely affecting training outcomes. To address this issue, we propose UIE with Diffusion Prior (UIEDP), a novel framework treating UIE as a posterior distribution sampling process of clear images conditioned on degraded underwater inputs. Specifically, UIEDP combines a pre-trained diffusion model capturing natural image priors with any existing UIE algorithm, leveraging the latter to guide conditional generation. The diffusion prior mitigates the drawbacks of inferior synthetic images, resulting in higher-quality image generation. Extensive experiments have demonstrated that our UIEDP yields significant improvements across various metrics, especially no-reference image quality assessment. And the generated enhanced images also exhibit a more natural appearance.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the insufficient quality of synthetic reference images in underwater image enhancement (UIE). Specifically:

1. **Problem Background**:
- Underwater image enhancement aims to generate clear images from low - quality underwater images.
- Due to the lack of real high - quality reference images, researchers usually construct paired datasets by synthetic methods to train deep - learning models.
- However, the quality of these synthetic images is sometimes low, resulting in poor training effects.

2. **Proposed Method**:
- The paper proposes a new framework named UIEDP (Underwater Image Enhancement with Diffusion Prior), which regards UIE as a posterior distribution sampling process for generating clear images under given degraded underwater input conditions.
- UIEDP combines a pre - trained diffusion model (capturing natural image priors) and any existing UIE algorithm, using the latter to guide conditional generation.
- The diffusion prior can reduce the negative impact caused by the poor quality of synthetic images, thereby generating higher - quality images.

3. **Core Contributions**:
- The UIEDP framework is proposed, which can fully utilize the generation priors of pre - trained diffusion models to generate higher - quality enhanced images.
- This framework is applicable to both supervised and unsupervised settings, demonstrating its flexibility.
- Experimental results show that UIEDP significantly outperforms existing methods in various quality assessment metrics, especially in non - reference image quality assessment.

4. **Technical Details**:
- A pre - trained diffusion model is used as a generative model to capture the prior probability distribution of natural images.
- Pseudo - label images generated by any UIE algorithm are used to guide conditional generation, and enhanced images are sampled from the posterior probability distribution.
- A matching function \( L_{\text{match}}(y, x_t) \) is introduced to quantify the matching degree between the observed underwater image \( y \) and the sampled image \( x_t \).
- Four different guiding methods are designed, and it is experimentally verified that the optimal guiding method is to guide \( \tilde{x}_0 \) in the natural image domain.

5. **Experimental Results**:
- Experiments on multiple public datasets (such as UIEB, U45) show that UIEDP outperforms other methods in almost all assessment metrics, especially outstanding in non - reference image quality assessment metrics.
- Visualization results also show that the enhanced images generated by UIEDP have a more natural appearance, especially when dealing with complex underwater scenes.

In conclusion, this paper solves the problem of insufficient quality of synthetic reference images by introducing diffusion priors, significantly improving the effect of underwater image enhancement.