Multi-Frame Super-Resolution Algorithm Based on a WGAN
Keqing Ning,Zhihao Zhang,Kai Han,Siyu Han,Xiqing Zhang
DOI: https://doi.org/10.1109/access.2021.3088128
IF: 3.9
2021-01-01
IEEE Access
Abstract:Image super-resolution reconstruction has been widely used in remote sensing, medicine and other fields. In recent years, due to the rise of deep learning research and the successful application of convolutional neural networks in the image field, the super-resolution reconstruction technology based on deep learning has also achieved great development. However, there are still some problems that need to be solved. For example, the current mainstream image super-resolution algorithms based on single or multiple frames pursue high performance indicators such as PSNR and SSIM, while the reconstructed image is relatively smooth and lacks many high-frequency details. It is not conducive to application in a real environment. To address such problem, this paper proposes a super-resolution reconstruction model of sequential images based on Generative Adversarial Networks (GAN). The proposed approach combines the registration module to fuse adjacent frames, effectively use the detailed information in multiple consecutive frames, and enhances the spatio-temporality of low-resolution images in sequential images. While the GAN was used to improve the effect of image high-frequency texture detail reconstruction, WGAN was introduced to optimize model training. The reconstruction results not only improved the PSNR and SSIM indexes but also reconstructed more high-frequency detail textures. Finally, in order to further improve the perception effect, an additional registration loss item RLT is introduced in the GAN network perception loss. Through extensive experiments, it shows that the model proposed in this paper effectively obtains the information between the sequence images. When the PSNR and SSIM indicators are improve, it can reconstruct better high-frequency texture details than the current advanced multi-frame algorithms.
computer science, information systems,telecommunications,engineering, electrical & electronic