Abstract:We exploit the similarities between Tikhonov regularization and Bayesian hierarchical models to propose a regularization scheme that acts like a distributed Tikhonov regularization where the amount of regularization varies from component to component. In the standard formulation, Tikhonov regularization compensates for the inherent ill-conditioning of linear inverse problems by augmenting the data fidelity term measuring the mismatch between the data and the model output with a scaled penalty functional. The selection of the scaling is the core problem in Tikhonov regularization. If an estimate of the amount of noise in the data is available, a popular way is to use the Morozov discrepancy principle, stating that the scaling parameter should be chosen so as to guarantee that the norm of the data fitting error is approximately equal to the norm of the noise in the data. A too small value of the regularization parameter would yield a solution that fits to the noise while a too large value would lead to an excessive penalization of the solution. In many applications, it would be preferable to apply distributed regularization, replacing the regularization scalar by a vector valued parameter, allowing different regularization for different components of the unknown, or for groups of them. A distributed Tikhonov-inspired regularization is particularly well suited when the data have significantly different sensitivity to different components, or to promote sparsity of the solution. The numerical scheme that we propose, while exploiting the Bayesian interpretation of the inverse problem and identifying the Tikhonov regularization with the Maximum A Posteriori (MAP) estimation, requires no statistical tools. A combination of numerical linear algebra and optimization tools makes the scheme computationally efficient and suitable for problems where the matrix is not explicitly available.

Stopping criterion for iterative regularization of large-scale ill-posed problems using the Picard parameter

A preconditioned Krylov subspace method for linear inverse problems with general-form Tikhonov regularization

A fast iterative regularization method for ill-posed problems

Scalable iterative data-adaptive RKHS regularization

Early Stopping for Ensemble Kalman-Bucy Inversion

A relaxed iterated Tikhonov regularization for linear ill-posed inverse problems

Hanke-Raus heuristic rule for iteratively regularized stochastic gradient descent

Early stopping for statistical inverse problems via truncated SVD estimation

Distributed Tikhonov regularization for ill-posed inverse problems from a Bayesian perspective

Stopping criteria for nonlinear variational problems: iterative approach

A Joint Bidiagonalization Based Algorithm for Large Scale Linear Discrete Ill-posed Problems in General-Form Regularization

Convergence analysis and parameter estimation for the iterated Arnoldi-Tikhonov method

Subspace projection regularization for large-scale Bayesian linear inverse problems

Iterative regularization for low complexity regularizers

Generalized hybrid iterative methods for large-scale Bayesian inverse problems

Projected Newton method for large-scale Bayesian linear inverse problems

Early stopping for conjugate gradients in statistical inverse problems

Uncertainty quantification for iterative algorithms in linear models with application to early stopping

A new stopping criterion for Krylov solvers applied in Interior Point Methods

Modified iterated Tikhonov methods for solving systems of nonlinear ill-posed equations

Parameter Selection by GCV and a $χ^2$ test within Iterative Methods for $\ell_1$-regularized Inverse Problems