Noise Conditional Flow Model for Learning the Super-Resolution Space

Younggeun Kim,Donghee Son

DOI: https://doi.org/10.48550/arXiv.2106.04428

2021-06-06

Abstract:Fundamentally, super-resolution is ill-posed problem because a low-resolution image can be obtained from many high-resolution images. Recent studies for super-resolution cannot create diverse super-resolution images. Although SRFlow tried to account for ill-posed nature of the super-resolution by predicting multiple high-resolution images given a low-resolution image, there is room to improve the diversity and visual quality. In this paper, we propose Noise Conditional flow model for Super-Resolution, NCSR, which increases the visual quality and diversity of images through noise conditional layer. To learn more diverse data distribution, we add noise to training data. However, low-quality images are resulted from adding noise. We propose the noise conditional layer to overcome this phenomenon. The noise conditional layer makes our model generate more diverse images with higher visual quality than other works. Furthermore, we show that this layer can overcome data distribution mismatch, a problem that arises in normalizing flow models. With these benefits, NCSR outperforms baseline in diversity and visual quality and achieves better visual quality than traditional GAN-based models. We also get outperformed scores at NTIRE 2021 challenge.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to generate more diverse and higher - visual - quality high - resolution images in the single - image super - resolution task. Specifically, although existing super - resolution methods can reconstruct high - resolution images from low - resolution images, these methods often fail to produce diverse results, especially when dealing with the inherent ill - posed nature of the super - resolution problem, that is, a low - resolution image may correspond to multiple high - resolution images. Although SRFlow attempts to solve this problem by predicting multiple high - resolution images, there is still room for improvement in terms of diversity and visual quality.

To this end, the paper proposes a method based on the Noise Conditional Flow Model for Super - Resolution (NCSR). By adding noise to the training data and introducing a noise conditional layer, it overcomes the problem of image quality degradation caused by simply adding noise. This method can not only generate more diverse images, but also improve the visual quality of the images, and at the same time solve the data distribution mismatch problem in the normalizing flow model.

The main contributions of the paper are as follows:
1. Propose a method to improve super - resolution space learning by adding noise and using a noise conditional layer.
2. Expand the data distribution by adding noise to the training data, thereby increasing diversity.
3. Solve the performance degradation problem caused by data distribution mismatch when using normalizing flow for super - resolution modeling.

Through these improvements, the NCSR model is superior to the baseline model in terms of diversity and visual quality, and has achieved excellent results in the NTIRE 2021 challenge.