Abstract:For the large-scale linear discrete ill-posed problem minAx-b or Ax=b with b contaminated by a white noise, Lanczos bidiagonalization based LSQR and its mathematically equivalent CGLS are most commonly used. They have intrinsic regularizing effects, where the number k of iterations plays the role of regularization parameter. However, hitherto there has been no answer to the long-standing fundamental concern of Björck and Eldén in 1979: for which kinds of problems LSQR and CGLS can find best possible regularized solutions? Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method or by standard-form Tikhonov regularization and cannot be improved under certain conditions. In this paper we make a detailed analysis on the regularization of LSQR for severely, moderately and mildly ill-posed problems. For the first two kinds of problems, we prove that LSQR finds best possible solutions at semi-convergence and the following results hold until semi-convergence: (i) the k-step Lanczos bidiagonalization always generates a near best rank k approximation to A; (ii) the k Ritz values always approximate the first k large singular values of A in natural order; (iii) the k-step LSQR always captures the k dominant SVD components of A; (iv) the diagonals and subdiagonals of the bidiagonal matrices generated by Lanczos bidiagonalization decay as fast as the singular values of A. However, for the third kind of problem, the above results do not hold generally. We also analyze the regularization of the other two Krylov solvers LSMR and CGME, proving that LSMR has similar regularizing effects to LSQR for each kind of problem and both are superior to CGME. Numerical experiments confirm our theory on LSQR.

The Low Rank Approximations and Ritz Values in LSQR for Linear Discrete Ill-Posed Problem*

The Krylov Subspaces, Low Rank Approximations and Ritz Values of LSQR for Linear Discrete Ill-Posed Problems: the Multiple Singular Value Case

Approximation Accuracy of the Krylov Subspaces for Linear Discrete Ill-Posed Problems

The Regularization Theory of the Krylov Iterative Solvers LSQR, CGLS, LSMR and CGME for Linear Discrete Ill-Posed Problems

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

N A ] 5 D ec 2 01 4 Some Results on Regularization of LSQR and CGLS for Large-Scale Discrete Ill-Posed Problems ∗

Regularization Properties of Krylov Iterative Solvers CGME and LSMR for Linear Discrete Ill-Posed Problems with an Application to Truncated Randomized SVDs

CONVERGENCE AND STABILITY OF ITERATIVELY REWEIGHTED LEAST SQUARES FOR LOW-RANK MATRIXRECOVERY

Some Properties of Lsqr for Large Sparse Linear Least Squares Problems

Double Precision is not Necessary for LSQR for Solving Discrete Linear Ill-Posed Problems

Modified Truncated Randomized Singular Value Decomposition (MTRSVD) Algorithms for Large Scale Discrete Ill-Posed Problems with General-Form Regularization

Generalizing the SVD of a matrix under non-standard inner product and its applications to linear ill-posed problems

On Convergence of the Inexact Rayleigh Quotient Iteration with the Lanczos Method Used for Solving Linear Systems

A Refined Harmonic Lanczos Bidiagonalization Method and an Implicitly Restarted Algorithm for Computing the Smallest Singular Triplets of Large Matrices

A Special Modified Tikhonov Regularization Matrix for Discrete Ill-Posed Problems

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

On Regularizing Effects of MINRES and MR-II for Large Scale Symmetric Discrete Ill-Posed Problems

Truncated LSQR for matrix least squares problems

A Joint Bidiagonalization Based Algorithm for Large Scale Linear Discrete Ill-posed Problems in General-Form Regularization

Preconditioned Low-Rank Riemannian Optimization for Symmetric Positive Definite Linear Matrix Equations

Hybrid LSMR algorithms for large-scale general-form regularization