MetaUE: Model-based Meta-learning for Underwater Image Enhancement

Zhenwei Zhang,Haorui Yan,Ke Tang,Yuping Duan

2023-03-12

Abstract:The challenges in recovering underwater images are the presence of diverse degradation factors and the lack of ground truth images. Although synthetic underwater image pairs can be used to overcome the problem of inadequately observing data, it may result in over-fitting and enhancement degradation. This paper proposes a model-based deep learning method for restoring clean images under various underwater scenarios, which exhibits good interpretability and generalization ability. More specifically, we build up a multi-variable convolutional neural network model to estimate the clean image, background light and transmission map, respectively. An efficient loss function is also designed to closely integrate the variables based on the underwater image model. The meta-learning strategy is used to obtain a pre-trained model on the synthetic underwater dataset, which contains different types of degradation to cover the various underwater environments. The pre-trained model is then fine-tuned on real underwater datasets to obtain a reliable underwater image enhancement model, called MetaUE. Numerical experiments demonstrate that the pre-trained model has good generalization ability, allowing it to remove the color degradation for various underwater attenuation images such as blue, green and yellow, etc. The fine-tuning makes the model able to adapt to different underwater datasets, the enhancement results of which outperform the state-of-the-art underwater image restoration methods. All our codes and data are available at \url{<a class="link-external link-https" href="https://github.com/Duanlab123/MetaUE" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

The problems that this paper attempts to solve are two major challenges faced in underwater image restoration: 1) the existence of multiple degradation factors, and 2) the lack of real images as references. Although the problem of insufficient data can be overcome by synthesizing underwater image pairs, this may lead to over - fitting and a decline in the enhancement effect. To solve these problems, the paper proposes a model - based deep - learning method for restoring clear images in various underwater scenarios, which has good interpretability and generalization ability.

Specifically, the paper constructs a multivariate convolutional neural network model to estimate the clear image, background light, and transmission map respectively. In addition, an effective loss function is designed to closely integrate these variables according to the underwater image model. Through the meta - learning strategy, a pre - trained model is obtained on a synthetic underwater data set containing different types of degradation to cover various underwater environments. Then, the pre - trained model is fine - tuned on the real underwater data set to obtain a reliable underwater image enhancement model, named MetaUE.

The experimental results show that the pre - trained model has good generalization ability and can remove the color degradation of various underwater attenuated images (such as blue, green, and yellow). Fine - tuning enables the model to adapt to different underwater data sets, and its enhancement results are better than the existing state - of - the - art underwater image restoration methods. All the code and data are available on GitHub: https://github.com/Duanlab123/MetaUE.