A PRIORI STRATEGY IN DISCRETIZATION OF THE TIKHONOV REGULARIZATION

J Cheng,M Yamamoto,J Zou

DOI: https://doi.org/10.1142/9789812794253_0158

2003-01-01

Abstract:We propose a strategy for a priori choice of regularizing parameters and discretization mesh sizes in the Tikhonov regularization for guaranteeing a convergence rate. Our strategy is based on the conditional stability estimate of the inverse problem under consideration.

What problem does this paper attempt to address?

One New Strategy For A Priori Choice Of Regularizing Parameters In Tikhonov'S Regularization

Jin Cheng,Masahiro Yamamoto

DOI: https://doi.org/10.1088/0266-5611/16/4/101

IF: 2.408

2000-01-01

Inverse Problems

Abstract:In this paper, based on the conditional stability estimate for ill-posed inverse problems, we propose a new strategy for a priori choice of regularizing parameters in Tikhonov's regularization and we show that it can be applied to a wide class of inverse problems. The convergence rate of the regularized solutions is also proved.
A Graph-based Iterative Algorithm for Inverse Problems with Local and Non-local Regularizations

Ming Chen,Chuan Xie,Yongchao Wang,Zhijie Lin,Lei Zhao

DOI: https://doi.org/10.2991/ifmca-16.2017.17

2017-01-01

Abstract:I In this paper, we propose a new form of prior which combines the local total variation regularization with non-local graph-based variation regularization to solve such inverse problems in the field of image processing. The algorithm iteratively updates the image and the weight graph calculated from the present image..
A discretizing Tikhonov regularization method via modified parameter choice rules

Rong Zhang,Feiping Xie,Xingjun Luo

DOI: https://doi.org/10.1515/jiip-2023-0056

IF: 1.4479

2024-01-06

Journal of Inverse and Ill-Posed Problems

Abstract:In this paper, we propose two parameter choice rules for the discretizing Tikhonov regularization via multiscale Galerkin projection for solving linear ill-posed integral equations. In contrast to previous theoretical analyses, we introduce a new concept called the projection noise level to obtain error estimates for the approximate solutions. This concept allows us to assess how noise levels change during projection. The balance principle and Hankeâ€“Raus rule are modified by incorporating the error estimates of the projection noise level. We demonstrate the convergence rate of these two modified parameter choice rules through rigorous proof. In addition, we find that the error between the approximate solution and the exact solution improves as the noise frequency increases. Finally, numerical experiments are provided to illustrate the theoretical findings presented in this paper.

mathematics, applied
An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Shuai Lu,Sergei V. Pereverzev,Ronny Ramlau

DOI: https://doi.org/10.1088/0266-5611/23/1/011

IF: 2.408

2006-01-01

Inverse Problems

Abstract:In this paper we introduce an adaptive regularization scheme based on algorithms for minimization of the Tikhonov functional to reconstruct the solution x(dagger) of nonlinear ill-posed problem F(x) = y, where the right-hand side is replaced by noisy data y(delta) is an element of Y with vertical bar vertical bar y - y(delta)vertical bar vertical bar <= delta, and F : D(F) subset of X -> Y is a nonlinear operator between Hilbert spaces X and Y. Under the assumption that Frechet derivative F ' is Lipschitz continuous, a choice of the regularization parameter and the stopping criteria for minimization algorithms are presented. We prove that under a general source condition given in terms of a nonlinear operator F the error vertical bar vertical bar x(dagger) - x(k)vertical bar vertical bar between the regularized approximation x(k) and the solution x(dagger) is of optimal order.
Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces

Rodrigo Maulen-Soto,Jalal Fadili,Hedy Attouch

2024-03-27

Abstract:To solve non-smooth convex optimization problems with a noisy gradient input, we analyze the global behavior of subgradient-like flows under stochastic errors. The objective function is composite, being equal to the sum of two convex functions, one being differentiable and the other potentially non-smooth. We then use stochastic differential inclusions where the drift term is minus the subgradient of the objective function, and the diffusion term is either bounded or square-integrable. In this context, under Lipschitz's continuity of the differentiable term and a growth condition of the non-smooth term, our first main result shows almost sure weak convergence of the trajectory process towards a minimizer of the objective function. Then, using Tikhonov regularization with a properly tuned vanishing parameter, we can obtain almost sure strong convergence of the trajectory towards the minimum norm solution. We find an explicit tuning of this parameter when our objective function satisfies a local error-bound inequality. We also provide a comprehensive complexity analysis by establishing several new pointwise and ergodic convergence rates in expectation for the convex and strongly convex case.

Optimization and Control
A relaxed iterated Tikhonov regularization for linear ill-posed inverse problems

Weike Chang,Nicola D'Ascenzo,Qingguo Xie

DOI: https://doi.org/10.1016/j.jmaa.2023.127754

IF: 1.417

2024-02-01

Journal of Mathematical Analysis and Applications

Abstract:Iterated Tikhonov regularization methods are prevalent regularization methods that can make the solution of ill-posed problems less sensitive to noise. As the iteration number grows, their solution gradually converges to the optimum approximation of the desired solution but then changes to the worse, causing a semiconvergence. This difficulty can be reduced by a reliable early termination rule or a strategy for choosing the appropriate regularization parameter at each iteration. Different from the existing methods, this paper describes a novel iterative Tikhonov regularization based on introducing a non-decreasing sequence of relaxation parameters into the stationary iterated Tikhonov regularization, providing a fast and roughly stable convergence. After investigating theoretically the convergence rate of the novel iterative Tikhonov regularization, we propose a series of numerical experiments for the evaluation of its accuracy and finally describe a real-world application in the field of 3-dimensional image restoration problems to verify its efficiency.

mathematics, applied
Multi-parameter Tikhonov regularization with the ` sparsity constraint: ` convergence

Shuai Lu

2014-01-01

Abstract:We establish ` error estimates for the infinite-dimensional multi-parameter Tikhonov regularization with ` and ` penalty terms. Lacking of interpolation inequalities within different sequences spaces (i.e. `, ` and `), one cannot directly obtain ` error estimates for the proposed multi-parameter Tikhonov functional where no ` information appears in the functional form. By building up the variational inequality in ` norm, we derive that both a priori and a posteriori parameter choice rules provide ` error estimates of optimal order under appropriate source conditions, and the corresponding constants can be verified in an explicit manner. In the latter case, we have implemented the classic and sequential discrepancy principles respectively. Finally an adopted prime-dual algorithm illustrates the sparsity promoting properties of the considered multi-parameter Tikhonov regularization. It is a jointed work with Wei Wang (Jiaxing University, China), Bernd Hofmann (TU Chemnitz, Germany) and Jin Cheng (Fudan University, China).
Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Hu Yaohua,Li Chong,Meng Kaiwen,Yang Xiaoqi

DOI: https://doi.org/10.1007/s10898-020-00955-3

2020-01-01

Journal of Global Optimization

Abstract:The ℓ _p regularization problem with 0< p< 1 has been widely studied for finding sparse solutions of linear inverse problems and gained successful applications in various mathematics and applied science fields. The proximal gradient algorithm is one of the most popular algorithms for solving the ℓ _p regularisation problem. In the present paper, we investigate the linear convergence issue of one inexact descent method and two inexact proximal gradient algorithms (PGA). For this purpose, an optimality condition theorem is explored to provide the equivalences among a local minimum, second-order optimality condition and second-order growth property of the ℓ _p regularization problem. By virtue of the second-order optimality condition and second-order growth property, we establish the linear convergence properties of the inexact descent method and inexact PGAs under some simple assumptions. Both linear convergence to a local minimal value and linear convergence to a local minimum are provided. Finally, the linear convergence results of these methods are extended to the infinite-dimensional Hilbert spaces. Our results cannot be established under the framework of Kurdyka–Łojasiewicz theory.
Tikhonov regularized mixed-order primal-dual dynamical system for convex optimization problems with linear equality constraints

Honglu Li,Xin He,Yibin Xiao

2024-09-27

Abstract:In Hilbert spaces, we consider a Tikhonov regularized mixed-order primal-dual dynamical system for a convex optimization problem with linear equality constraints. The dynamical system with general time-dependent parameters: viscous damping and temporal scaling can derive certain existing systems when special parameters are selected. When these parameters satisfy appropriate conditions and the Tikhonov regularization parameter \epsilon(t) approaches zero at an appropriate rate, we analyze the asymptotic convergence properties of the proposed system by constructing suitable Lyapunov functions. And we obtain that the objective function error enjoys O(1/(t^2\beta(t))) convergence rate. Under suitable conditions, it can be better than O(1/(t^2)). In addition, we utilize the Lyapunov analysis method to obtain the strong convergence of the trajectory generated by the Tikhonov regularized dynamical system. In particular, when Tikhonov regularization parameter \epsilon(t) vanishes to 0 at some suitable rate, the convergence rate of the primal-dual gap can be o(1/(\beta(t))). The effectiveness of our theoretical results has been demonstrated through numerical experiments.

Optimization and Control
A preconditioned Krylov subspace method for linear inverse problems with general-form Tikhonov regularization

Haibo Li

2023-08-12

Abstract:Tikhonov regularization is a widely used technique in solving inverse problems that can enforce prior properties on the desired solution. In this paper, we propose a Krylov subspace based iterative method for solving linear inverse problems with general-form Tikhonov regularization term $x^TMx$, where $M$ is a positive semi-definite matrix. An iterative process called the preconditioned Golub-Kahan bidiagonalization (pGKB) is designed, which implicitly utilizes a proper preconditioner to generate a series of solution subspaces with desirable properties encoded by the regularizer $x^TMx$. Based on the pGKB process, we propose an iterative regularization algorithm via projecting the original problem onto small dimensional solution subspaces. We analyze regularization effect of this algorithm, including the incorporation of prior properties of the desired solution into the solution subspace and the semi-convergence behavior of regularized solution. To overcome instabilities caused by semi-convergence, we further propose two pGKB based hybrid regularization algorithms. All the proposed algorithms are tested on both small-scale and large-scale linear inverse problems. Numerical results demonstrate that these iterative algorithms exhibit excellent performance, outperforming other state-of-the-art algorithms in some cases.

Numerical Analysis
On the Generalized Discrepancy Principle for Tikhonov Regularization in Hilbert Scales

S. Lu,S. V. Pereverzev,Y. Shao,U. Tautenhahn

DOI: https://doi.org/10.1216/jie-2010-22-3-483

IF: 0.943

2010-01-01

Journal of Integral Equations and Applications

Abstract:For solving linear ill-posed problems regularization methods are required when the right hand side and the operator are with some noise. In the present paper regularized solutions are obtained by Tikhonov regularization in Hilbert scales and the regularization parameter is chosen by the generalized discrepancy principle. Under certain smoothness assumptions we provide order optimal error bounds that characterize the accuracy of the regularized solution. It appears that for getting small error bounds a proper scaling of the penalizing operator B is required. For the computation of the regularization parameter fast algorithms of Newton type are constructed which are based on special transformations. These algorithms are globally and monotonically convergent. The results extend earlier results where the problem operator is exactly given. Some of our theoretical results are illustrated by numerical experiments.
Fast convergence rates and trajectory convergence of a Tikhonov regularized inertial primal\mbox{-}dual dynamical system with time scaling and vanishing damping

Ting-Ting Zhu,Rong Hu,Ya-Ping Fang

2024-04-23

Abstract:A Tikhonov regularized inertial primal\mbox{-}dual dynamical system with time scaling and vanishing damping is proposed for solving a linearly constrained convex optimization problem in Hilbert spaces. The system under consideration consists of two coupled second order differential equations and its convergence properties depend upon the decaying speed of the product of the time scaling parameter and the Tikhonov regularization parameter (named the rescaled regularization parameter) to zero. When the rescaled regularization parameter converges rapidly to zero, the system enjoys fast convergence rates of the primal-dual gap, the feasibility violation, the objective residual, and the gradient norm of the objective function along the trajectory, and the weak convergence of the trajectory to a primal-dual solution of the linearly constrained convex optimization problem. When the rescaled regularization parameter converges slowly to zero, the generated primal trajectory converges strongly to the minimal norm solution of the problem under suitable conditions. Finally, numerical experiments are performed to illustrate the theoretical findings.

Optimization and Control
The Index Function and Tikhonov Regularization for Ill-Posed Problems

Jin Cheng,Bernd Hofmann,Shuai Lu

DOI: https://doi.org/10.1016/j.cam.2013.09.035

IF: 2.872

2013-01-01

Journal of Computational and Applied Mathematics

Abstract:In this paper, we study the regularizing properties of the conditional stability estimates in ill-posed problems. First, we analyze how conditional stability estimates occur, and which properties the corresponding index functions must obey. In addition, we adapt the convergence analysis for the Tikhonov regularization in Banach spaces where the difference between the approximated solution and the exact one in metric measure is taken into account. We conclude this study with a comparison of stability estimates and variational inequalities, another emerging tool in Banach space regularization.
Convergence and adaptive discretization of the IRGNM Tikhonov and the IRGNM Ivanov method under a tangential cone condition in Banach space

Barbara Kaltenbacher,Mario Luiz Previatti de Souza

DOI: https://doi.org/10.1007/s00211-018-0971-5

Abstract:In this paper we consider the iteratively regularized Gauss-Newton method (IRGNM) in its classical Tikhonov version as well as two further-Ivanov type and Morozov type-versions. In these two alternative versions, regularization is achieved by imposing bounds on the solution or by minimizing some regularization functional under a constraint on the data misfit, respectively. We do so in a general Banach space setting and under a tangential cone condition, while convergence (without source conditions, thus without rates) has so far only been proven under stronger restrictions on the nonlinearity of the operator and/or on the spaces. Moreover, we provide a convergence result for the discretized problem with an appropriate control on the error and show how to provide the required error bounds by goal oriented weighted dual residual estimators. The results are illustrated for an inverse source problem for a nonlinear elliptic boundary value problem, for the cases of a measure valued and of an L∞ source. For the latter, we also provide numerical results with the Ivanov type IRGNM.
Small noise analysis for Tikhonov and RKHS regularizations

Quanjun Lang,Fei Lu

2024-09-04

Abstract:Regularization plays a pivotal role in ill-posed machine learning and inverse problems. However, the fundamental comparative analysis of various regularization norms remains open. We establish a small noise analysis framework to assess the effects of norms in Tikhonov and RKHS regularizations, in the context of ill-posed linear inverse problems with Gaussian noise. This framework studies the convergence rates of regularized estimators in the small noise limit and reveals the potential instability of the conventional L2-regularizer. We solve such instability by proposing an innovative class of adaptive fractional RKHS regularizers, which covers the L2 Tikhonov and RKHS regularizations by adjusting the fractional smoothness parameter. A surprising insight is that over-smoothing via these fractional RKHSs consistently yields optimal convergence rates, but the optimal hyper-parameter may decay too fast to be selected in practice.

Machine Learning
A Tikhonov Regularization Based Algorithm for Scattered Data with Random Noise

Jiantang Zhang,Jin Cheng,Min Zhong

DOI: https://doi.org/10.48550/arXiv.2105.00747

2021-05-03

Numerical Analysis

Abstract:With the rapid growth of data, how to extract effective information from data is one of the most fundamental problems. In this paper, based on Tikhonov regularization, we propose an effective method for reconstructing the function and its derivative from scattered data with random noise. Since the noise level is not assumed small, we will use the amount of data for reducing the random error, and use a relatively small number of knots for interpolation. An indicator function for our algorithm is constructed. It indicates where the numerical results are good or may not be good. The corresponding error estimates are obtained. We show how to choose the number of interpolation knots in the reconstruction process for balancing the random errors and interpolation errors. Numerical examples show the effectiveness and rapidity of our method. It should be remarked that the algorithm in this paper can be used for on-line data.
Well-posedness and Tikhonov regularization of an inverse source problem for a parabolic equation with an integral constraint

Sedar Ngoma

DOI: https://doi.org/10.1515/jiip-2023-0050

IF: 1.4479

2024-04-03

Journal of Inverse and Ill-Posed Problems

Abstract:We investigate a time-dependent inverse source problem for a parabolic partial differential equation with an integral constraint and subject to Neumann boundary conditions in a domain of R d \mathbb{R}^{d} , d ‚Č• 1 d\geq 1 . We prove the well-posedness as well as higher regularity of solutions in H√∂lder spaces. We then develop and implement an algorithm that we use to approximate solutions of the inverse problem by means of a finite element discretization in space. Due to instability in inverse problems, we apply Tikhonov regularization combined with the discrepancy principle for selecting the regularization parameter in order to obtain a stable reconstruction. Our numerical results show that the proposed scheme is an accurate technique for approximating solutions of this inverse problem.

mathematics, applied
Multi-parameter Regularization and Its Numerical Realization

Shuai Lu,Sergei V. Pereverzev

DOI: https://doi.org/10.1007/s00211-010-0318-3

IF: 2.5

2010-01-01

Numerische Mathematik

Abstract:In this paper we propose and analyse a choice of parameters in the multi-parameter regularization of Tikhonov type. A modified discrepancy principle is presented within the multi-parameter regularization framework. An order optimal error bound is obtained under the standard smoothness assumptions. We also propose a numerical realization of the multi-parameter discrepancy principle based on the model function approximation. Numerical experiments on a series of test problems support theoretical results. Finally we show how the proposed approach can be successfully implemented in Laplacian Regularized Least Squares for learning from labeled and unlabeled examples.
Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

Ting-Ting Zhu,Rong Hu,Ya-Ping Fang

DOI: https://doi.org/10.1080/02331934.2024.2407515

IF: 2.2

2024-09-26

Optimization

Abstract:In this paper, in the setting of Hilbert spaces, we consider a Tikhonov regularized second-order plus first-order primal-dual dynamical system with asymptotically vanishing damping for a linear equality constrained convex optimization problem. It is shown that convergence properties of the proposed inertial dynamical system depend upon the choice of the Tikhonov regularization parameter. When the Tikhonov regularization parameter decays rapidly to zero, we derive the fast convergence rates of the primal-dual gap, the objective residual, the feasibility violation and the gradient norm of the objective function along the generated trajectory. When the Tikhonov regularization parameter decreases slowly to zero, we prove the strong convergence of the primal trajectory of the Tikhonov regularized dynamical system to the minimal norm solution of the linear equality constrained convex optimization problem. Numerical experiments are performed to illustrate the efficiency of our approach.

mathematics, applied,operations research & management science
Automatic nonstationary anisotropic Tikhonov regularization through bilevel optimization

Silvia Gazzola,Ali Gholami

2024-06-04

Abstract:Regularization techniques are necessary to compute meaningful solutions to discrete ill-posed inverse problems. The well-known 2-norm Tikhonov regularization method equipped with a discretization of the gradient operator as regularization operator penalizes large gradient components of the solution to overcome instabilities. However, this method is homogeneous, i.e., it does not take into account the orientation of the regularized solution and therefore tends to smooth the desired structures, textures and discontinuities, which often contain important information. If the local orientation field of the solution is known, a possible way to overcome this issue is to implement local anisotropic regularization by penalizing weighted directional derivatives. In this paper, considering problems that are inherently two-dimensional, we propose to automatically and simultaneously recover the regularized solution and the local orientation parameters (used to define the anisotropic regularization term) by solving a bilevel optimization problem. Specifically, the lower level problem is Tikhonov regularization equipped with local anisotropic regularization, while the objective function of the upper level problem encodes some natural assumptions about the local orientation parameters and the Tikhonov regularization parameter. Application of the proposed algorithm to a variety of inverse problems in imaging (such as denoising, deblurring, tomography and Dix inversion), with both real and synthetic data, shows its effectiveness and robustness.

Numerical Analysis

A PRIORI STRATEGY IN DISCRETIZATION OF THE TIKHONOV REGULARIZATION

One New Strategy For A Priori Choice Of Regularizing Parameters In Tikhonov'S Regularization

A Graph-based Iterative Algorithm for Inverse Problems with Local and Non-local Regularizations

A discretizing Tikhonov regularization method via modified parameter choice rules

An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces

A relaxed iterated Tikhonov regularization for linear ill-posed inverse problems

Multi-parameter Tikhonov regularization with the ` sparsity constraint: ` convergence

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Tikhonov regularized mixed-order primal-dual dynamical system for convex optimization problems with linear equality constraints

A preconditioned Krylov subspace method for linear inverse problems with general-form Tikhonov regularization

On the Generalized Discrepancy Principle for Tikhonov Regularization in Hilbert Scales

Fast convergence rates and trajectory convergence of a Tikhonov regularized inertial primal\mbox{-}dual dynamical system with time scaling and vanishing damping

The Index Function and Tikhonov Regularization for Ill-Posed Problems

Convergence and adaptive discretization of the IRGNM Tikhonov and the IRGNM Ivanov method under a tangential cone condition in Banach space

Small noise analysis for Tikhonov and RKHS regularizations

A Tikhonov Regularization Based Algorithm for Scattered Data with Random Noise

Well-posedness and Tikhonov regularization of an inverse source problem for a parabolic equation with an integral constraint

Multi-parameter Regularization and Its Numerical Realization

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

Automatic nonstationary anisotropic Tikhonov regularization through bilevel optimization