Abstract:The objective of this paper is to understand the superlinear convergence behavior of the inner-iteration preconditioned GMRES method. In order to understand the phenomenon, we analyze the convergence using the Vandermonde matrix which is defined using the eigenvalues of the coefficient matrix. Although eigenvalues alone cannot explain the convergence, they may provide an upper bound of the residual, together with the right hand side vector and the eigenvectors. For the diagonalizable case, if the eigenvalues of the coefficient matrix are clustered, the upper bound of the convergence curve shows superlinear convergence, when the norm of the matrix obtained by decomposing the right hand side vector into the eigenvector components is not so large. We especially analyze the effect of inner-iteration preconditioning for least squares problems, where the eigenvalues cluster towards 1.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to understand the super - linear convergence behavior of the inner - iteration preconditioned GMRES method. Specifically, the author analyzes the possibility of using the Vandermonde matrix (defined by the eigenvalues of the coefficient matrix) to explain this phenomenon. Although the individual eigenvalues cannot fully explain the convergence, they can, together with the right - hand - side vector and eigenvectors, provide an upper bound for the residual. For the diagonalizable case, if the eigenvalues of the coefficient matrix are clustered together and the norm of the matrix obtained by decomposing the right - hand - side vector into eigenvector components is not very large, the upper bound of the convergence curve will exhibit super - linear convergence.
In addition, the author specifically analyzes the effect of inner - iteration preconditioning in the least - squares problem, where the eigenvalues tend to 1. Through these analyses, the author hopes to reveal how inner - iteration preconditioning accelerates the convergence of the GMRES method.
### Formula Summary
- **Vandermonde Matrix**:
\[
\Lambda_k^d=\begin{pmatrix}
\lambda_1&\lambda_2&\cdots&\lambda_d\\
\lambda_1^2&\lambda_2^2&\cdots&\lambda_d^2\\
\vdots&\vdots&\ddots&\vdots\\
\lambda_1^k&\lambda_2^k&\cdots&\lambda_d^k
\end{pmatrix}\in\mathbb{C}^{d\times k}
\]
- **Inner - Iteration Preconditioned GMRES Method for the Least - Squares Problem**:
\[
\min_{\tilde{x}_k = \tilde{x}_0+\tilde{z}_k,\tilde{z}_k\in K_k(\tilde{A},\tilde{r}_0)}\|\tilde{b}-\tilde{A}\tilde{x}_k\|_2
\]
where \(K_k(\tilde{A},\tilde{r}_0)=\text{span}\{\tilde{r}_0,\tilde{A}\tilde{r}_0,\ldots,\tilde{A}^{k - 1}\tilde{r}_0\}\) is the Krylov subspace.
- **Upper Bound Estimation of the Residual**:
\[
\|\tilde{b}-\tilde{A}\tilde{x}_k\|_2\leq\|V\text{diag}[c_1,c_2,\ldots,c_d]\|_2\min_{y\in\mathbb{C}^k}\|\Lambda_k^d y - [1,1,\ldots,1]^T\|_2
\]
### Key Conclusions
1. **Effect of Eigenvalue Clustering**: When the eigenvalues are clustered, especially when the eigenvalues in the least - squares problem tend to 1, the convergence speed of the GMRES method will be significantly accelerated.
2. **Role of Inner - Iteration Preconditioning**: Inner - iteration preconditioning can improve the distribution of eigenvalues, thereby accelerating the convergence of the GMRES method.
3. **Super - Linear Convergence Behavior**: By analyzing the influence of eigenvalues and eigenvectors, the author proves that the inner - iteration preconditioned GMRES method can exhibit super - linear convergence in some cases.
These results provide theoretical support for understanding the convergence behavior of the GMRES method and guidance for optimization in practical applications.