Adaptive optimal regularization of the linear ill posed integral equations

E.Ostrovsky L.Sirota

DOI: https://doi.org/10.48550/arXiv.1004.0662

2010-04-06

Abstract:We construct an adaptive asymptotically optimal in order in the $ L(2) $ sense a solution (estimation) of an integral linear equation of a first kind and energy of this solution with the confidence region building, also adaptive.

Numerical Analysis,Statistics Theory

What problem does this paper attempt to address?

Optimization of Approximate Solution of Integral Equations of Several Variables

Wan Ma,Xing Hua Wang

DOI: https://doi.org/10.1007/s10114-002-0175-9

2002-01-01

Acta Mathematica Sinica English Series

Abstract:In this paper, we consider the problem of optimization of adaptive direct methods of operator equations. Adaptivity of a direct method is understood in the sense that the subspace on the basis of which it is constructed is chosen depending on the operator of the concrete equation (otherwise, nonadaptive direct method is then concerned), which would essentially let us increase the precision. For some classes of the second kind of Fredhlom integral equations with anisotropic smooth kernels we determine the exact order of the error of adaptive direct methods, and we also give an optimal algorithm.
Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov Optimizing Control

Fernando Pazos,Amit Bhaya

DOI: https://doi.org/10.1016/j.cam.2014.10.022

IF: 2.872

2015-01-01

Journal of Computational and Applied Mathematics

Abstract:Numerical problems, including linear algebraic equations, are frequently ill-posed, which means that small perturbations in the observation data can have large effects on the computed solutions. In these cases some kind of regularization method must be used to find a feasible solution. Regularization implies the modification of the original ill-posed problem, so that it becomes well-posed, and the Tikhonov regularization method is the most often used. This method requires the choice of a regularization parameter, and this choice is a crucial issue. Recently, several papers were presented proposing adaptive Tikhonov regularization parameters, instead of constant ones. This paper revisits all these adaptive choices and shows that they can be interpreted in a unified and systematic manner using the Liapunov Optimizing Control (LOC) method, which, in addition, opens up the possibility of new choices of the regularization parameter. Examples of matrices from real applications show that one of the methods designed by the LOC method, called the OVM method by its discoverer, can even outperform the CG method for some matrices, although the CG method performs better for a larger class of matrices.
An asymptotical regularization with convex constraints for inverse problems

Min Zhong,Wei Wang,Shanshan Tong

DOI: https://doi.org/10.1088/1361-6420/ac55ef

IF: 2.408

2022-03-08

Inverse Problems

Abstract:Abstract We investigate the method of asymptotical regularization for the stable approximate solution of nonlinear ill-posed problems F ( x ) = y in Hilbert spaces. The method consists of two components, an outer Newton iteration and an inner scheme providing increments by solving a local coupling linearized evolution equations. In addition, a non-smooth uniformly convex functional has been embedded in the evolution equations which is allowed to be non-smooth, including L 1 -liked and total variation-like penalty terms. We establish convergence properties of the method, derive stability estimates, and perform the convergence rate under the Hölder continuity of the inverse mapping. Furthermore, based on Runge–Kutta (RK) discretization, different kinds of iteration schemes can be developed for numerical realization. In our numerical experiments, four types iterative scheme, including Landweber type, one-stage explicit, implicit Euler and two-stage RK are presented to illustrate the performance of the proposed method.

mathematics, applied,physics, mathematical
An adaptive RKHS regularization for Fredholm integral equations

Fei Lu,Miao-Jung Yvonne Ou

2023-12-05

Abstract:Regularization is a long-standing challenge for ill-posed linear inverse problems, and a prototype is the Fredholm integral equation of the first kind with additive Gaussian measurement noise. We introduce a new RKHS regularization adaptive to measurement data and the underlying linear operator. This RKHS arises naturally in a variational approach, and its closure is the function space in which we can identify the true solution. Also, we introduce a small noise analysis to compare regularization norms by sharp convergence rates in the small noise limit. Our analysis shows that the RKHS- and $L^2$-regularizers yield the same convergence rate when their optimal hyper-parameters are selected using the true solution, and the RKHS-regularizer has a smaller multiplicative constant. However, in computational practice, the RKHS regularizer significantly outperforms the $L^2$-and $l^2$-regularizers in producing consistently converging estimators when the noise level decays or the observation mesh refines.

Numerical Analysis
A novel iterative integration regularization method for ill-posed inverse problems

Ce Huang,Li Wang,Minghui Fu,Zhong-Rong Lu,Yanmao Chen

DOI: https://doi.org/10.1007/s00366-019-00920-z

IF: 8.083

2020-01-08

Engineering With Computers

Abstract:This paper proposes a new iterative integration regularization method for robust solution of ill-posed inverse problems. The proposed method is motivated from the fact that inversion of a positive definite matrix can be expressed in an integral form. Then, the development of the proposed method is mainly twofold. Firstly, two ways—including the linear iteration and the exponential ((2^j)) iteration—are invoked to compute the integral, of which the exponential iteration is often preferred due to its fast convergence. Secondly, after stability analysis, the proposed method is shown able to filter out the undesired effect of relatively small singular values, while preserving the desired terms of relatively large singular values, i.e., the proposed method has the guaranteed regularization effect. Numerical examples on three typical ill-posed problems are conducted with detailed comparison to some usual direct and iterative regularization methods. Final results have highlighted the proposed method: (a) due to the iterative nature, the proposed method often turns out to be more efficient than the conventional direct regularization methods including the Tikhonov regularization and the truncated singular value decomposition (TSVD), (b) the proposed method converges much faster than the Landweber method and (c) the regularization effect is guaranteed in the proposed method, while may not be in the conjugate gradient method for least squares problem (CGLS).

engineering, mechanical,computer science, interdisciplinary applications
An adaptive finite element method for distributed elliptic optimal control problems with variable energy regularization

Ulrich Langer,Richard Löscher,Olaf Steinbach,Huidong Yang

2023-08-14

Abstract:We analyze the finite element discretization of distributed elliptic optimal control problems with variable energy regularization, where the usual $L^2(\Omega)$ norm regularization term with a constant regularization parameter $\varrho$ is replaced by a suitable representation of the energy norm in $H^{-1}(\Omega)$ involving a variable, mesh-dependent regularization parameter $\varrho(x)$. It turns out that the error between the computed finite element state $\widetilde{u}_{\varrho h}$ and the desired state $\overline{u}$ (target) is optimal in the $L^2(\Omega)$ norm provided that $\varrho(x)$ behaves like the local mesh size squared. This is especially important when adaptive meshes are used in order to approximate discontinuous target functions. The adaptive scheme can be driven by the computable and localizable error norm $\| \widetilde{u}_{\varrho h} - \overline{u}\|_{L^2(\Omega)}$ between the finite element state $\widetilde{u}_{\varrho h}$ and the target $\overline{u}$. The numerical results not only illustrate our theoretical findings, but also show that the iterative solvers for the discretized reduced optimality system are very efficient and robust.

Numerical Analysis,Optimization and Control
A fast iterative regularization method for ill-posed problems

Tahar Bechouat

DOI: https://doi.org/10.1007/s10092-024-00626-9

2024-11-28

CALCOLO

Abstract:Ill-posed problems manifest in a wide range of scientific and engineering disciplines. The solutions to these problems exhibit a high degree of sensitivity to data perturbations. Regularization methods strive to alleviate the sensitivity exhibited by these solutions. This paper presents a fast iterative scheme for addressing linear ill-posed problems, similar to nonstationary iterated Tikhonov regularization. Both the a-priori and a-posteriori choice rules for regularization parameters are provided, and both rules yield error estimates that are order optimal. In comparison to the nonstationary iterated Tikhonov method, the newly introduced method significantly reduces the required number of iterations to achieve convergence based on an appropriate stopping criterion. The numerical computations provide compelling evidence regarding the efficacy of our new iterative regularization method. Furthermore, the versatility of this method extends to image restorations.

mathematics, applied
Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Hu Yaohua,Li Chong,Meng Kaiwen,Yang Xiaoqi

DOI: https://doi.org/10.1007/s10898-020-00955-3

2020-01-01

Journal of Global Optimization

Abstract:The ℓ _p regularization problem with 0< p< 1 has been widely studied for finding sparse solutions of linear inverse problems and gained successful applications in various mathematics and applied science fields. The proximal gradient algorithm is one of the most popular algorithms for solving the ℓ _p regularisation problem. In the present paper, we investigate the linear convergence issue of one inexact descent method and two inexact proximal gradient algorithms (PGA). For this purpose, an optimality condition theorem is explored to provide the equivalences among a local minimum, second-order optimality condition and second-order growth property of the ℓ _p regularization problem. By virtue of the second-order optimality condition and second-order growth property, we establish the linear convergence properties of the inexact descent method and inexact PGAs under some simple assumptions. Both linear convergence to a local minimal value and linear convergence to a local minimum are provided. Finally, the linear convergence results of these methods are extended to the infinite-dimensional Hilbert spaces. Our results cannot be established under the framework of Kurdyka–Łojasiewicz theory.
A New Highly Anti-Interference Regularization Method for Ill-Posed Problems

Hai Liang Zhuang,Hai Bo Chen

DOI: https://doi.org/10.21595/vp.2017.19358

2017-01-01

Vibroengineering PROCEDIA

Abstract:The solution of inverse problems has many applications in mathematical physics. Regularization methods can be applied to obtain the solution of ill-conditioned inverse problems by solving a family of neighboring well-posed problems. Thus, it is significant to investigate the regularization methods to increase the accuracy and efficiency of the solution of inverse problems. In this work, a new regularization filter and the related regularization method based on the singular system theory of compact operator are proposed to solve ill-posed problems. The Cauchy problem of Laplace equation of the first kind is a kind of well-known ill-posed problem. Numerical tests show that the proposed regularization method can solve the Cauchy problems more efficiently under a proper selection of regularization parameters. Numerical results also show that the proposed method is especially effective in solving ill-posed problems with big perturbations.
An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Shuai Lu,Sergei V. Pereverzev,Ronny Ramlau

DOI: https://doi.org/10.1088/0266-5611/23/1/011

IF: 2.408

2006-01-01

Inverse Problems

Abstract:In this paper we introduce an adaptive regularization scheme based on algorithms for minimization of the Tikhonov functional to reconstruct the solution x(dagger) of nonlinear ill-posed problem F(x) = y, where the right-hand side is replaced by noisy data y(delta) is an element of Y with vertical bar vertical bar y - y(delta)vertical bar vertical bar <= delta, and F : D(F) subset of X -> Y is a nonlinear operator between Hilbert spaces X and Y. Under the assumption that Frechet derivative F ' is Lipschitz continuous, a choice of the regularization parameter and the stopping criteria for minimization algorithms are presented. We prove that under a general source condition given in terms of a nonlinear operator F the error vertical bar vertical bar x(dagger) - x(k)vertical bar vertical bar between the regularized approximation x(k) and the solution x(dagger) is of optimal order.
Iterative regularization for low complexity regularizers

Molinari, Cesare

DOI: https://doi.org/10.1007/s00211-023-01390-8

IF: 2.5

2024-02-11

Numerische Mathematik

Abstract:Iterative regularization exploits the implicit bias of optimization algorithms to regularize ill-posed problems. Constructing algorithms with such built-in regularization mechanisms is a classic challenge in inverse problems but also in modern machine learning, where it provides both a new perspective on algorithms analysis, and significant speed-ups compared to explicit regularization. In this work, we propose and study the first iterative regularization procedure with explicit computational steps able to handle biases described by non smooth and non strongly convex functionals, prominent in low-complexity regularization. Our approach is based on a primal-dual algorithm of which we analyze convergence and stability properties, even in the case where the original problem is unfeasible. The general results are illustrated considering the special case of sparse recovery with the penalty. Our theoretical results are complemented by experiments showing the computational benefits of our approach.

mathematics, applied
Adaptive $$l_p$$ $$(0<p<1)$$ Regularization: Oracle Property and Applications

Yunxiao Shi,Xiangnan He,Han Wu,Zhong-Xiao Jin,Wenlian Lu

DOI: https://doi.org/10.1007/978-3-319-70087-8_2

2017-01-01

Abstract:In this paper, we propose adaptive Lp (0 < p < 1) estimators in sparse, high-dimensional, linear regression models when the number of covariates depends on the sample size. Other than the case of the number of covariates is smaller than the sample size, in this paper, we prove that under appropriate conditions, these adaptive Lp estimators possess the oracle property in the case that the number of covariates is much larger than the sample size. We present a series of experiments demonstrating the remarkable performance of this estimator with adaptive Lp regularization, in comparison with the L1 regularization, the adaptive L1 regularization, and non-adaptive Lp regularization with 0 < p < 1, and its broad applicability in variable selection, signal recovery and shape reconstruction.
Influence of the order between discretization and regularization in solving ill-posed problems

Laurence Grammont,Paulo B. Vasconcelos

DOI: https://doi.org/10.1016/j.matcom.2024.02.010

IF: 3.601

2024-02-01

Mathematics and Computers in Simulation

Abstract:Discretization and regularization are required steps to provide a stable approximation when solving integral equations of the first kind. The integral operator involved may be approximated by a sequence of finite rank operators and then the regularization procedure is applied. On the other hand, a regularization procedure can be conceived prior to the discretization. Both approaches are developed, implemented and compared for certain projection based methods.

mathematics, applied,computer science, software engineering, interdisciplinary applications
Regularization with two differential operators and its application to inverse problems

Shuang Yu,Hongqi Yang

DOI: https://doi.org/10.1016/j.apnum.2024.02.017

IF: 2.994

2024-03-03

Applied Numerical Mathematics

Abstract:We introduce a regularization method with two differential operators for solving a linear ill-posed operator equation system. The existence and uniqueness of regularized solutions to the problem are derived. With an a priori as well as an a posteriori parameter choice strategy, convergence and convergence rates of the regularized solution are also obtained. As an application, we apply the regularization to a simultaneous inversion of the source term and the initial value problem for a heat conduction equation, numerical experiments are presented to demonstrate the effectiveness of the proposed method.

mathematics, applied
Interval regularization for imprecise linear algebraic equations

Sergey P. Shary

DOI: https://doi.org/10.48550/arXiv.1810.01481

2018-09-27

Abstract:In this paper, we consider the solution of ill-conditioned systems of linear algebraic equations that can be determined imprecisely. To improve the stability of the solution process, we "immerse" the original imprecise linear system in an interval system of linear algebraic equations of the same structure and then consider its tolerable solution set. As the result, the "intervalized" matrix of the system acquires close and better conditioned matrices for which the solution of the corresponding equation system is more stable. As a pseudo-solution of the original linear equation system, we take a point from the tolerable solution set of the intervalized linear system or a point that provides the largest tolerable compatibility (consistency). We propose several computational recipes to find such pseudo-solutions.

Numerical Analysis
On Learning the Optimal Regularization Parameter in Inverse Problems

Jonathan Chirinos Rodriguez,Ernesto De Vito,Cesare Molinari,Lorenzo Rosasco,Silvia Villa

2024-05-21

Abstract:Selecting the best regularization parameter in inverse problems is a classical and yet challenging problem. Recently, data-driven approaches have become popular to tackle this challenge. These approaches are appealing since they do require less a priori knowledge, but their theoretical analysis is limited. In this paper, we propose and study a statistical machine learning approach, based on empirical risk minimization. Our main contribution is a theoretical analysis, showing that, provided with enough data, this approach can reach sharp rates while being essentially adaptive to the noise and smoothness of the problem. Numerical simulations corroborate and illustrate the theoretical findings. Our results are a step towards grounding theoretically data-driven approaches to inverse problems.

Statistics Theory,Optimization and Control
Adaptive Methods or Variational Inequalities with Relatively Smooth and Reletively Strongly Monotone Operators

S. S. Ablaev,F. S. Stonyakin,M. S. Alkousa,D. A. Pasechnyuk

2023-08-01

Abstract:The article is devoted to some adaptive methods for variational inequalities with relatively smooth and relatively strongly monotone operators. Starting from the recently proposed proximal variant of the extragradient method for this class of problems, we investigate in detail the method with adaptively selected parameter values. An estimate of the convergence rate of this method is proved. The result is generalized to a class of variational inequalities with relatively strongly monotone generalized smooth variational inequality operators. Numerical experiments have been performed for the problem of ridge regression and variational inequality associated with box-simplex games.

Optimization and Control
Optimal regularized hypothesis testing in statistical inverse problems

Remo Kretschmann,Daniel Wachsmuth,Frank Werner

DOI: https://doi.org/10.1088/1361-6420/ad1132

2023-10-17

Abstract:Testing of hypotheses is a well studied topic in mathematical statistics. Recently, this issue has also been addressed in the context of Inverse Problems, where the quantity of interest is not directly accessible but only after the inversion of a (potentially) ill-posed operator. In this study, we propose a regularized approach to hypothesis testing in Inverse Problems in the sense that the underlying estimators (or test statistics) are allowed to be biased. Under mild source-condition type assumptions we derive a family of tests with prescribed level $\alpha$ and subsequently analyze how to choose the test with maximal power out of this family. As one major result we prove that regularized testing is always at least as good as (classical) unregularized testing. Furthermore, using tools from convex optimization, we provide an adaptive test by maximizing the power functional, which then outperforms previous unregularized tests in numerical simulations by several orders of magnitude.

Statistics Theory,Numerical Analysis
On the order optimality of the regularization via inexact Newton iterations

Qinian Jin

DOI: https://doi.org/10.48550/arXiv.1111.1883

2011-11-08

Abstract:Inexact Newton regularization methods have been proposed by Hanke and Rieder for solving nonlinear ill-posed inverse problems. Every such a method consists of two components: an outer Newton iteration and an inner scheme providing increments by regularizing local linearized equations. The method is terminated by a discrepancy principle. In this paper we consider the inexact Newton regularization methods with the inner scheme defined by Landweber iteration, the implicit iteration, the asymptotic regularization and Tikhonov regularization. Under certain conditions we obtain the order optimal convergence rate result which improves the suboptimal one of Rieder. We in fact obtain a more general order optimality result by considering these inexact Newton methods in Hilbert scales.

Numerical Analysis

Adaptive optimal regularization of the linear ill posed integral equations

Optimization of Approximate Solution of Integral Equations of Several Variables

Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov Optimizing Control

An asymptotical regularization with convex constraints for inverse problems

An adaptive RKHS regularization for Fredholm integral equations

A novel iterative integration regularization method for ill-posed inverse problems

An adaptive finite element method for distributed elliptic optimal control problems with variable energy regularization

A fast iterative regularization method for ill-posed problems

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A New Highly Anti-Interference Regularization Method for Ill-Posed Problems

An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Iterative regularization for low complexity regularizers

Adaptive $$l_p$$ $$(0<p<1)$$ Regularization: Oracle Property and Applications

Influence of the order between discretization and regularization in solving ill-posed problems

Regularization with two differential operators and its application to inverse problems

Interval regularization for imprecise linear algebraic equations

On Learning the Optimal Regularization Parameter in Inverse Problems

Adaptive Methods or Variational Inequalities with Relatively Smooth and Reletively Strongly Monotone Operators

Optimal regularized hypothesis testing in statistical inverse problems

On the order optimality of the regularization via inexact Newton iterations