Abstract:Super-resolution is the process of obtaining a high-resolution image from one or more low-resolution images. Single image super-resolution (SISR) and multi-frame super-resolution (MFSR) methods have been evolved almost independently for years. A neglected study in this field is the theoretical analysis of finding the optimum combination of SISR and MFSR. To fill this gap, we propose a novel theoretical analysis based on the iterative shrinkage and thresholding algorithm. We implement and compare several approaches for combining SISR and MFSR, and simulation results support the finding of our theoretical analysis, both quantitatively and qualitatively.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper attempts to solve the problem of how to optimally combine single - image super - resolution (SISR) and multi - frame image super - resolution (MFSR) methods. Specifically, through theoretical analysis and experimental verification, the author explores the best combination of SISR and MFSR methods under different up - sampling factors.
### Background and motivation
Super - resolution technology aims to restore high - resolution (HR) images from one or more low - resolution (LR) images. SISR and MFSR methods have been developed independently in the past few years, but few studies have theoretically analyzed how to optimally combine these two methods. The paper points out that although the SISR method can be regarded as a special case of the MFSR method, in practical applications, there are significant differences in their algorithms and tools. In addition, the super - resolution problem is essentially ill - posed, especially in SISR, where prior knowledge of the image is required to solve the under - determined problem.
### Methods and contributions
1. **Theoretical analysis**:
- The author proposes a new theoretical analysis method based on the Iterative Shrinkage and Thresholding Algorithm (ISTA).
- By deriving the fixed - point equation of the ISTA method, the author proves that the Multi - First Single - Last (MFSL) combination method is optimal.
2. **Experimental verification**:
- The author implements various combinations of SISR and MFSR methods and verifies the correctness of the theoretical analysis through a large number of experiments.
- The experimental results show that the MFSL combination method can achieve the best performance regardless of whether the up - sampling factor is divisible or not.
### Main conclusions
1. **When the up - sampling factor is indivisible**:
- When the up - sampling factor is an indivisible integer (such as 2 or 3), the MFSL combination method is optimal.
- Both theoretical analysis and experimental results support this conclusion.
2. **When the up - sampling factor is divisible**:
- When the up - sampling factor is a divisible integer (such as 4), the MFSL combination method is still optimal.
- The experimental results show that the MFSL method performs well under different data sets and noise levels.
### Experimental results
- **Performance comparison under different up - sampling factors**:
- For up - sampling factors 2 and 3, the MFSL combination method outperforms other methods under different data sets and noise levels.
- As the number of LR images increases, the performance gradually improves, but there is a saturation trend.
- **Verification on real data sets**:
- Tests were carried out on two real data sets (the Alpaca data set and a self - collected data set with almost no noise), and the results further verified the high performance of the MFSL combination method.
### Formula summary
1. **Image degradation model**:
\[
y_k = R F_k H x+w_k
\]
where \( x\in\mathbb{R}^n \) and \( y_k\in\mathbb{R}^m \) represent the HR image and the \( k \) - th LR image respectively, and \( R \), \( F_k \) and \( H \) represent the down - sampling, translation and blurring matrices respectively, and \( w_k \) represents additive white Gaussian noise.
2. **Cost function**:
\[
\rho(z)=\sum_{k = 1}^N\|R F_k z - y_k\|^2
\]
3. **Optimal solution**:
\[
\hat{z}=\sum_{k = 1}^N F_k^T R^T y_k
\]
4. **Sparse coding problem**:
\[
y = D_L\alpha
\]
\[
\hat{x}=D_H\alpha