A Single Image High-Perception Super-Resolution Reconstruction Method Based on Multi-layer Feature Fusion Model with Adaptive Compression and Parameter Tuning

Rui Zhang,Wenyu Ren,Lihu Pan,Xiaolu Bai,Ji Li
DOI: https://doi.org/10.1007/s11063-024-11660-7
IF: 2.565
2024-06-21
Neural Processing Letters
Abstract:We propose a simple image high-perception super-resolution reconstruction method based on multi-layer feature fusion model with adaptive compression and parameter tuning. The aim is to further balance the high and low-frequency information of an image, enrich the detailed texture to improve perceptual quality, and improve the adaptive optimization and generalization of the model in the process of super-resolution reconstruction. First, an effective multi-layer fusion super-resolution (MFSR) basic model is constructed by the design of edge enhancement, refine layering, enhanced super-resolution generative adversarial network and other sub-models, and effective multi-layer fusion. This further enriches the image representation of features of different scales and depths and improves the feature representation of high and low-frequency information in a balanced way. Next, a total loss function of the generator is constructed with adaptive parameter tuning performance. The overall adaptability of the model is improved through adaptive weight distribution and fusion of content loss, perceptual loss, and adversarial loss, and improving the error while reducing the edge enhancement model. Finally, a fitness function with the evaluation perceptual function as the optimization strategy is constructed, and the model compression and adaptive tuning of MFSR are carried out based on the multi-mechanism fusion strategy. Consequently, the construction of the adaptive MFSR model is realized. Adaptive MFSR can maintain high peak signal to noise ratio and structural similarity on the test sets Set5, Set14, and BSD100, and achieve high-quality reconstructed images with low learned perceptual image patch similarity and perceptual index, while having good generalization capabilities.
computer science, artificial intelligence
What problem does this paper attempt to address?
The problems that this paper attempts to solve are several key challenges in high - perceptual single - image super - resolution reconstruction. Specifically, these problems include: 1. **Balance between high - frequency and low - frequency information**: Most of the existing methods focus on deep feature extraction of single - scale images and lack representation of different - scale and depth features, resulting in missing or blurry edge details in the reconstructed image, which affects the image quality. In addition, the balanced reconstruction effect of high - frequency and low - frequency information needs to be improved, and the perceptual quality still needs to be optimized. 2. **Subjectivity in network structure compression and parameter optimization**: Current super - resolution network structure compression and parameter optimization mainly rely on the subjective experience of developers, which requires professional experience and reliable domain knowledge, and optimizes the network structure through a large number of time - consuming "trial - and - error" experiments to obtain better reconstruction results. It lacks the self - adaptive optimization performance of the model and often has problems such as structural redundancy and weak model generalization ability. To solve the above problems, this research proposes an adaptive high - perceptual super - resolution reconstruction method based on multi - layer feature fusion (AMFSR). The main contributions of this method include: 1. **Constructing a multi - layer fused super - resolution model (MFSR)**: Through single - layer design and multi - layer effective fusion of sub - models such as edge enhancement, refinement layering, and ESRGAN, the image feature representation of different scales and depths is further enriched, thereby comprehensively improving the feature representation of high - frequency and low - frequency information in the image. 2. **Constructing a generator total loss function with adaptive tuning performance**: Through adaptive weight distribution and effective fusion of content loss, perceptual loss, and adversarial loss, the overall adaptability of the model is improved. 3. **Global optimization strategy based on multi - mechanism fusion strategy**: A global optimization strategy based on multi - mechanism fusion strategy (MFS) is designed. The constructed perceptual function (PF) is used as the fitness function of the optimization strategy and is successfully applied to the adaptive compression of the MFSR model structure and the adaptive adjustment of key super - parameters, ultimately achieving the construction of the AMFSR model, improving the generalization ability of the model while achieving model compression optimization. These improvements aim to improve the image quality and perceptual quality of super - resolution reconstruction, especially in terms of edge details and overall image quality.