Abstract:For the large-scale linear discrete ill-posed problem $\min\|Ax-b\|$ or $Ax=b$ with $b$ contaminated by white noise, the Golub-Kahan bidiagonalization based LSQR method and its mathematically equivalent CGLS, the Conjugate Gradient (CG) method applied to $A^TAx=A^Tb$, are most commonly used. They have intrinsic regularizing effects, where the iteration number $k$ plays the role of regularization parameter. The long-standing fundamental question is: {\em Can LSQR and CGLS find 2-norm filtering best possible regularized solutions}? The author has given definitive answers to this question for severely and moderately ill-posed problems when the singular values of $A$ are simple. This paper extends the results to the multiple singular value case, and studies the approximation accuracy of Krylov subspaces, the quality of low rank approximations generated by Golub-Kahan bidiagonalization and the convergence properties of Ritz values. For the two kinds of problems, we prove that LSQR finds 2-norm filtering best possible regularized solutions at semi-convergence. Particularly, we consider some important and untouched issues on best, near best and general rank $k$ approximations to $A$ for the ill-posed problems with the singular values $\sigma_k=\mathcal{O}(k^{-\alpha})$ with $\alpha>0$, and the relationships between them and their nonzero singular values. Numerical experiments confirm our theory. The results on general rank $k$ approximations and the properties of their nonzero singular values apply to several Krylov solvers, including LSQR, CGME, MINRES, MR-II, GMRES and RRGMRES.

Lifting Sylvester equations: singular value decay for non-normal coefficients

Krylov methods for low-rank commuting generalized Sylvester equations

On Sylvester equations in Banach subalgebras

Lifting Brownian-type operators with subnormal entry

A new projection method for solving large Sylvester equations

The Krylov Subspaces, Low Rank Approximations and Ritz Values of LSQR for Linear Discrete Ill-Posed Problems: the Multiple Singular Value Case

Nonharmonic multivariate Fourier transforms and matrices: condition numbers and hyperplane geometry

Generalised rank-constrained approximations of Hilbert-Schmidt operators on separable Hilbert spaces and applications

A nested divide-and-conquer method for tensor Sylvester equations with positive definite hierarchically semiseparable coefficients

The Low Rank Approximations and Ritz Values in LSQR for Linear Discrete Ill-Posed Problem*

Krylov Subspace Methods for the Generalized Sylvester Equation.

An SVD-like Decomposition of Bounded-Input Bounded-Output Functions

A note on the Krylov solvability of compact normal operators on Hilbert space

Improved Spectral Density Estimation via Explicit and Implicit Deflation

Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions

The Bilinear Bochner-Riesz Problem

Solving Abel integral equations by regularisation in Hilbert scales

A general theory of singular values with applications to signal denoising

Spectral Condition-Number Estimation of Large Sparse Matrices

Regularization of linear ill-posed problems involving multiplication operators

Injective ellipticity, cancelling operators, and endpoint Gagliardo-Nirenberg-Sobolev inequalities for vector fields