A Regularization Method for Treating Zero Points of the Sum of Two Monotone Operators

Xiaolong Qin,Sun Young Cho,Lin Wang

DOI: https://doi.org/10.1186/1687-1812-2014-75

2014-01-01

Fixed Point Theory and Applications

Abstract:In this paper, a regularization method for treating zero points of the sum of two monotone operators is investigated. Strong convergence theorems are established in the framework of Hilbert spaces.

What problem does this paper attempt to address?

A Unified Convergence Theory for Newton-Type Methods for Zeros of Nonlinear Operators in Banach Spaces

Xinghua Wang,Chong Li,Ming-Jun Lai

DOI: https://doi.org/10.1023/A:1021986506085

IF: 1.856

2002-01-01

BIT Numerical Mathematics

Abstract:The paper is concerned with the convergence problem of Newton type methods for finding zeros of nonlinear operators in Banach spaces. Some families of nonlinear operators redefined by different Lipschitz conditions and an “universal constant” is introduced so that unified convergence determination of these methods is established for the defined families.
Regularization with two differential operators and its application to inverse problems

Shuang Yu,Hongqi Yang

DOI: https://doi.org/10.1016/j.apnum.2024.02.017

IF: 2.994

2024-03-03

Applied Numerical Mathematics

Abstract:We introduce a regularization method with two differential operators for solving a linear ill-posed operator equation system. The existence and uniqueness of regularized solutions to the problem are derived. With an a priori as well as an a posteriori parameter choice strategy, convergence and convergence rates of the regularized solution are also obtained. As an application, we apply the regularization to a simultaneous inversion of the source term and the initial value problem for a heat conduction equation, numerical experiments are presented to demonstrate the effectiveness of the proposed method.

mathematics, applied
An Uni ed Convergence Theory for Newton's Type Methods for Zeros of Nonlinear Operators in Banach spaces

Xinghua Wang,Chong Li

2007-01-01

Abstract:The paper is concerned with the convergence problem of Newton's type methods for nding zeros of nonlinear operators in Banach spaces. Some families of nonlinear operators are de ned by di erent Lipschitz conditions and an \universal constant" is introduced so that an uni ed convergence determination of these methods is established for the de ned families.
A second order dynamical system method for solving a maximal comonotone inclusion problem

Zengzhen Tan,Rong Hu,Yaping Fang

2023-07-07

Abstract:In this paper a second order dynamical system model is proposed for computing a zero of a maximal comonotone operator in Hilbert spaces. Under mild conditions, we prove existence and uniqueness of a strong global solution of the proposed dynamical system. A proper tuning of the parameters can allow us to establish fast convergence properties of the trajectories generated by the dynamical system. The weak convergence of the trajectory to a zero of the maximal comonotone operator is also proved. Furthermore, a discrete version of the dynamical system is considered and convergence properties matching to that of the dynamical system are established under a same framework. Finally, the validity of the proposed dynamical system and its discrete version is demonstrated by two numerical examples.

Optimization and Control
Strong convergence of new iterative projection methods with regularization for solving monotone variational inequalities in Hilbert spaces

Dang Van Hieu,Le Dung Muu,Hoang Ngoc Duong,Buu Huu Thai

DOI: https://doi.org/10.1002/mma.6647

2020-07-09

Mathematical Methods in the Applied Sciences

Abstract:<p>In this paper, we introduce two new numerical methods for solving a variational inequality problem involving a monotone and Lipschitz continuous operator in a Hilbert space. We describe how to incorporate a regularization term depending on a parameter in the projection method and then establish the strong convergence of the resulting iterative regularization projection methods. Unlike known hybrid methods, the strong convergence of the new methods comes from the regularization technique. The first method is designed to work in the case where the Lipschitz constant of cost operator is known, whereas the second one is more easily implemented without this requirement. The reason is because the second method has used a simple computable stepsize rule. The variable stepsizes are generated by the second method at each iteration and based on the previous iterates. These stepsizes are found with only one cheap computation without line‐search procedure. Several numerical experiments are implemented to show the computational effectiveness of the new methods over existing methods.</p>

mathematics, applied
Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Jinhua Wang,Chong Li,Genaro Lopez,Jen-Chih Yao

DOI: https://doi.org/10.1137/15m1051257

IF: 2.763

2016-01-01

SIAM Journal on Optimization

Abstract:In the present paper, we consider inexact proximal point algorithms for finding singular points of multivalued vector fields on Hadamard manifolds. The rate of convergence is shown to be linear under the mild assumption of metric subregularity. Furthermore, if the sequence of parameters associated with the iterative scheme converges to 0, then the convergence rate is superlinear. At the same time, the finite termination of the inexact proximal point algorithm is also provided under a weak sharp minima-like condition. Applications to optimization problems are provided. Some of our results are new even in Euclidean spaces, while others improve and/or extend some known results in Euclidean spaces. As a matter of fact, in the case of exact proximal point algorithm, our results improve the corresponding results in [G. C. Bento and J. X. Cruz Neto, Opttm., 63 (2014), pp. 1281-1288]. Finally, several examples are provided to illustrate that our results are applicable while the corresponding results in the Hilbert space setting are not.
Monotone vector fields and the proximal point algorithm on Hadamard manifolds

C. Li,G. Lopez,V. Martin-Marquez

DOI: https://doi.org/10.1112/jlms/jdn087

2009-01-01

Abstract:The maximal monotonicity notion in Banach spaces is extended to Riemannian manifolds of nonpositive sectional curvature, Hadamard manifolds, and proved to be equivalent to the upper semicontinuity. We consider the problem of finding a singularity of a multivalued vector field in a Hadamard manifold and present a general proximal point method to solve that problem, which extends the known proximal point algorithm in Euclidean spaces. We prove that the sequence generated by our method is well defined and converges to a singularity of a maximal monotone vector field, whenever it exists. Applications in minimization problems with constraints, minimax problems and variational inequality problems, within the framework of Hadamard manifolds, are presented.
Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Jinhua Wang,Chong Li,K. F. Ng

DOI: https://doi.org/10.1137/22M152147X

2023-01-01

Abstract:We study the issue of strong convergence of inexact proximal point algorithms (introduced by Rockafellar in [SIAM J. Control Optim., 14 (1976), pp. 877-898]) for maximal monotone operators on Hilbert spaces. A unified global/local strong convergence of inexact proximal point algorithms is established under the Holder metrically subregular condition. Furthermore, quantitative estimates on the convergence rate of inexact proximal point algorithms are also provided. Applying to the special case of the classical (exact) proximal point algorithm, our results improve the corresponding ones in [G. Li and B. S. Mordukhovich, SIAM J. Optim., 22 (2012), pp. 1655-1684]. Finally, as applications, global/local strong convergence and estimates on the convergence rate of inexact proximal point algorithms for optimization problems are presented.
Third Order Dynamical Systems for the Sum of Two Generalized Monotone Operators

DOI: https://doi.org/10.1007/s10957-024-02437-y

2024-06-03

Journal of Optimization Theory and Applications

Abstract:In this paper, we propose and analyze a third-order dynamical system for finding zeros of the sum of two generalized operators in a Hilbert space . We establish the existence and uniqueness of the trajectories generated by the system under appropriate continuity conditions, and prove exponential convergence to the unique zero when the sum of the operators is strongly monotone. Additionally, we derive an explicit discretization of the dynamical system, which results in a forward–backward algorithm with double inertial effects and larger range of stepsize. We establish the linear convergence of the iterates to the unique solution using this algorithm. Furthermore, we provide convergence analysis for the class of strongly pseudo-monotone variational inequalities. We illustrate the effectiveness of our approach by applying it to structured optimization and pseudo-convex optimization problems.

mathematics, applied,operations research & management science
On the proximal point algorithms for solving the monotone inclusion problem

Tao Zhang,Shiru Li,Yong Xia

2023-12-22

Abstract:We consider finding a zero point of the maximally monotone operator $T$. First, instead of using the proximal point algorithm (PPA) for this purpose, we employ PPA to solve its Yosida regularization $T_{\lambda}$. Then, based on an $O(a_{k+1})$ ($a_{k+1}\geq \varepsilon>0$) resolvent index of $T$, it turns out that we can establish a convergence rate of $O (1/{\sqrt{\sum_{i=0}^{k}a_{i+1}^2}})$ for both the $\|T_{\lambda}(\cdot)\|$ and the gap function $\mathtt{Gap}(\cdot)$ in the non-ergodic sense, and $O(1/\sum_{i=0}^{k}a_{i+1})$ for $\mathtt{Gap}(\cdot)$ in the ergodic sense. Second, to enhance the convergence rate of the newly-proposed PPA, we introduce an accelerated variant called the Contracting PPA. By utilizing a resolvent index of $T$ bounded by $O(a_{k+1})$ ($a_{k+1}\geq \varepsilon>0$), we establish a convergence rate of $O(1/\sum_{i=0}^{k}a_{i+1})$ for both $\|T_{\lambda}(\cdot)\|$ and $\mathtt {Gap}(\cdot)$, considering the non-ergodic sense. Third, to mitigate the limitation that the Contracting PPA lacks a convergence guarantee, we propose two additional versions of the algorithm. These novel approaches not only ensure guaranteed convergence but also provide sublinear and linear convergence rates for both $\|T_{\lambda}(\cdot)\|$ and $\mathtt {Gap}(\cdot)$, respectively, in the non-ergodic sense.

Optimization and Control
A reconsideration on convergence of the extra-gradient method for solving quasimonotone variational inequalities

Li-Jun Zhu,Tzu-Chien Yin

DOI: https://doi.org/10.1080/02331934.2024.2341941

IF: 2.2

2024-04-13

Optimization

Abstract:In this paper, we continue to investigate convergence of the extra-gradient method for solving quasimonotone variational inequalities in Hilbert spaces. Under some suitable conditions, we prove that the iterative sequence generated by the extra-gradient method converges weakly to a solution of quasimonotone variational inequalities. Our result corrects and improves the main result in Salahuddin [The extra-gradient method for quasimonotone variational inequalities. Optimization. 2022;71:2519–2528].

mathematics, applied,operations research & management science
A New Approach to Solving the Split Common Solution Problem for Monotone Operator Equations in Hilbert Spaces

Nguyen Song Ha,Truong Minh Tuyen,Phan Thi Van Huyen

DOI: https://doi.org/10.1007/s00574-024-00405-8

2024-06-06

Bulletin of the Brazilian Mathematical Society New Series

Abstract:In the present paper, we propose a new approach to solving a class of generalized split problems. This approach will open some new directions for research to solve the other split problems, for instance, the split common zero point problem and the split common fixed point problem. More precisely, we study the split common solution problem for monotone operator equations in real Hilbert spaces. To find a solution to this problem, we propose and establish the strong convergence of the two new iterative methods by using the Tikhonov regularization method. Meantime, we also study the stability of the iterative methods. Finally, two numerical examples are also given to illustrate the effectiveness of the proposed methods.

mathematics
An Iterative Threshold Algorithm of Log-sum Regularization for Sparse Problem

Xin Zhou,Xiaowen Liu,Gong Zhang,Luliang Jia,Xu Wang,Zhiyuan Zhao

DOI: https://doi.org/10.1109/tcsvt.2023.3247944

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:The log-sum function as a penalty has always been drawing widespread attention in the field of sparse problems. However, it brings a non-convex, non-smooth and non-Lipschitz optimization problem that is difficult to tackle. To overcome the problem, an iterative threshold algorithm for the sparse optimization problems with log-sum function is proposed in this paper. For brevity, the sparse optimization problems with log-sum function are named log-sum regularization. Firstly, by introducing an intermediate function to construct another new function, a property theorem about solution for log-sum regularization is established. Secondly, based on the above theorem, the optimal setting rules of the compromising parameters are elaborated, and an iterative log-sum threshold algorithm is proposed. Thirdly, under the situation that the compromising parameters of log-sum regularization are relatively small, it can be proven that the proposed algorithm converges to a local minimizer of log-sum regularization. Finally, a series of simulations are implemented to examine performance of the proposed algorithm, and the results exhibit that the proposed algorithm outperforms the state-of-the-art algorithms.

engineering, electrical & electronic
A Relaxed Inertial Method for Solving Monotone Inclusion Problems with Applications

Chunxiang Zong,Yuchao Tang,Guofeng Zhang

DOI: https://doi.org/10.3390/sym16040466

2024-04-11

Symmetry

Abstract:We study a relaxed inertial forward–backward–half-forward splitting approach with variable step size to solve a monotone inclusion problem involving a maximal monotone operator, a cocoercive operator, and a monotone Lipschitz operator. The convergence of the sequence of iterations generated by the discretisations of a continuous-time dynamical system is established under suitable conditions. Given the challenges associated with computing the resolvent of the composite operator, the proposed method is employed to tackle the composite monotone inclusion problem. Additionally, a convergence analysis is conducted under certain conditions. To demonstrate the effectiveness of the algorithm, numerical experiments are performed on the image deblurring problem.

multidisciplinary sciences
A Stable Solution of a Nonuniformly Perturbed Quadratic Minimization Problem by the Extragradient Method with Step Size Separated from Zero

Artem'eva, L. A.,Dryazhenkov, A. A.,Potapov, M. M.

DOI: https://doi.org/10.1134/s0081543824030027

2024-08-23

Proceedings of the Steklov Institute of Mathematics

Abstract:A quadratic minimization problem is considered in Hilbert spaces under constraints given by a linear operator equation and a convex quadratic inequality. The main feature of the problem statement is that the practically available approximations to the exact linear operators specifying the criterion and the constraints converge to them only strongly pointwise rather than in the uniform operator norm, which makes it impossible to justify the use of the classical regularization methods. We propose a regularization method that is applicable in the presence of error estimates for approximate operators in pairs of other operator norms, which are weaker than the original ones. For each of the operators, the pair of corresponding weakened operator norms is obtained by strengthening the norm in the domain of the operator and weakening the norm in its range. The weakening of operator norms usually makes it possible to estimate errors in operators where this was fundamentally impossible in the original norms, for example, in the finite-dimensional approximation of a noncompact operator. From the original optimization formulation, a transition is made to the problem of finding a saddle point of the Lagrange function. The proposed numerical method for finding a saddle point is an iterative regularized extragradient two-stage procedure. At the first stage of each iteration, an approximation to the optimal value of the criterion is refined; at the second stage, the approximate solution with respect to the main variable is refined. Compared to the methods previously developed by the authors and working under similar information conditions, this method is preferable for practical implementation, since it does not require the gradient step size to converge to zero. The main result of the work is the proof of the strong convergence of the approximations generated by the method to one of the exact solutions to the original problem in the norm of the original space.

mathematics, applied
Tikhonov Regularization with L^0-Term Complementing a Convex Penalty: L^1 Convergence under Sparsity Constraints

Wei Wang,Shuai Lu,Bernd Hofmann,Jin Cheng

DOI: https://doi.org/10.1515/jiip-2019-0008

IF: 1.4479

2019-01-01

Journal of Inverse and Ill-Posed Problems

Abstract:Measuring the error by an l^1-norm, we analyze under sparsity assumptions an l^0-regularization approach, where the penalty in the Tikhonov functional is complemented by a general stabilizing convex functional. In this context, ill-posed operator equations Ax = y with an injective and bounded linear operator A mapping between l^2 and a Banach space Y are regularized. For sparse solutions, error estimates as well as linear and sublinear convergence rates are derived based on a variational inequality approach, where the regularization parameter can be chosen either a priori in an appropriate way or a posteriori by the sequential discrepancy principle. To further illustrate the balance between the l^0-term and the complementing convex penalty, the important special case of the l^2-norm square penalty is investigated showing explicit dependence between both terms. Finally, some numerical experiments verify and illustrate the sparsity promoting properties of corresponding regularized solutions.
Primal-dual extrapolation methods for monotone inclusions under local Lipschitz continuity

Zhaosong Lu,Sanyou Mei

2024-08-31

Abstract:In this paper we consider a class of monotone inclusion (MI) problems of finding a zero of the sum of two monotone operators, in which one operator is maximal monotone while the other is {\it locally Lipschitz} continuous. We propose primal-dual extrapolation methods to solve them using a point and operator extrapolation technique, whose parameters are chosen by a backtracking line search scheme. The proposed methods enjoy an operation complexity of ${\cal O}(\log \epsilon^{-1})$ and ${\cal O}(\epsilon^{-1}\log \epsilon^{-1})$, measured by the number of fundamental operations consisting only of evaluations of one operator and resolvent of the other operator, for finding an $\varepsilon$-residual solution of strongly and non-strongly MI problems, respectively. The latter complexity significantly improves the previously best operation complexity ${\cal O}(\varepsilon^{-2})$. As a byproduct, complexity results of the primal-dual extrapolation methods are also obtained for finding an $\varepsilon$-KKT or $\varepsilon$-residual solution of convex conic optimization, conic constrained saddle point, and variational inequality problems under {\it local Lipschitz} continuity. We provide preliminary numerical results to demonstrate the performance of the proposed methods.

Optimization and Control,Machine Learning,Numerical Analysis
On a Stochastic Differential Equation with Correction Term Governed by a Monotone and Lipschitz Continuous Operator

Radu Ioan Bot,Chiara Schindler

2024-04-28

Abstract:In our pursuit of finding a zero for a monotone and Lipschitz continuous operator $M : \R^n \rightarrow \R^n$ amidst noisy evaluations, we explore an associated differential equation within a stochastic framework, incorporating a correction term. We present a result establishing the existence and uniqueness of solutions for the stochastic differential equations under examination. Additionally, assuming that the diffusion term is square-integrable, we demonstrate the almost sure convergence of the trajectory process $X(t)$ to a zero of $M$ and of $\|M(X(t))\|$ to $0$ as $t \rightarrow +\infty$. Furthermore, we provide ergodic upper bounds and ergodic convergence rates in expectation for $\|M(X(t))\|^2$ and $\langle M(X(t), X(t)-x^*\rangle$, where $x^*$ is an arbitrary zero of the monotone operator. Subsequently, we apply these findings to a minimax problem. Finally, we analyze two temporal discretizations of the continuous-time models, resulting in stochastic variants of the Optimistic Gradient Descent Ascent and Extragradient methods, respectively, and assess their convergence properties.

Optimization and Control,Probability
Modified Tseng Method for Solving Pseudomonotone Variational Inequality Problem in Banach Spaces

Rose Maluleka,Godwin Chidi Ugwunnadi,Maggie Aphane,Hammed A. Abass,Abdul Rahim Khan

DOI: https://doi.org/10.3390/sym16030363

2024-03-19

Symmetry

Abstract:This article examines the process for solving the fixed-point problem of Bregman strongly nonexpansive mapping as well as the variational inequality problem of the pseudomonotone operator. Within the context of p-uniformly convex real Banach spaces that are also uniformly smooth, we introduce a modified Halpern iterative technique combined with an inertial approach and Tseng methods for finding a common solution of the fixed-point problem of Bregman strongly nonexpansive mapping and the pseudomonotone variational inequality problem. Using our iterative approach, we develop a strong convergence result for approximating the solution of the aforementioned problems. We also discuss some consequences of our major finding. The results presented in this paper complement and build upon many relevant discoveries in the literature.

multidisciplinary sciences
The dependence of local regularity of solutions on the summability of coefficients and nonhomogenous term

Zheng Li Bin Guo

2024-02-11

Abstract:In this paper, we mainly discuss the local regularity of the solution to the following problem \begin{align*} \begin{cases} -\dive({\bf{A}}(x)\nabla u(x))=f(x),&~x\in\Omega,\\ u(x)=0,&~x\in\partial\Omega, \end{cases} \end{align*} where $\Omega$ is a bounded domain in $\mathbb{R}^{n}$. In particular, we are concerned with the connection between the regularity of the solution $u$ and the integrability of the coefficient matrix ${\bf{A}}(x)$ as well as the nonhomogeneous term $f$. To be more precise, our first result is to prove that the maximum norm of $u$ can be controlled by $\|f\|_{s}$ with $f\in L^s(\Omega),~s>\frac{nq}{2q-n},~q>\frac{n}{2}$. Meanwhile, we construct some counterexamples to illustrate the index $\frac{nq}{2q-n}$ being sharp. Subsequently, we give an improved upper bound for the maximum norm of $u$. Namely, there exists a positive constant $C$ such that $$\|u\|_{\infty}\leq C\|f\|_{\frac{nq}{2q-n}}\left[\log\left(\frac{\|f\|_{s}}{~~~~~\|f\|_{\frac{nq}{2q-n}}}+1\right)+1\right].$$ Specially, the main difference of our approach compared to the arguments of [\ref{CUR}, \ref{XU}] is to construct two classes of truncation functions to remove the assumption of the boundedness of $u$. Finally, based on the previous results and Moser iteration argument, we derive the Harnack inequality of $u$ from which the Hölder continuity of the solution follows. In addition, we also find that the Lebesgue space $L^{\frac{n}{2}}(\Omega)$ to which the inverse of the smallest eigenvalue $\lambda(x) $ of the matrix {\bf{A}}(x) belongs is essentially sharp in order to establish local boundedness and the Hölder continuity of the solution.

Analysis of PDEs

A Regularization Method for Treating Zero Points of the Sum of Two Monotone Operators

A Unified Convergence Theory for Newton-Type Methods for Zeros of Nonlinear Operators in Banach Spaces

Regularization with two differential operators and its application to inverse problems

An Uni ed Convergence Theory for Newton's Type Methods for Zeros of Nonlinear Operators in Banach spaces

A second order dynamical system method for solving a maximal comonotone inclusion problem

Strong convergence of new iterative projection methods with regularization for solving monotone variational inequalities in Hilbert spaces

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Monotone vector fields and the proximal point algorithm on Hadamard manifolds

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Third Order Dynamical Systems for the Sum of Two Generalized Monotone Operators

On the proximal point algorithms for solving the monotone inclusion problem

A reconsideration on convergence of the extra-gradient method for solving quasimonotone variational inequalities

A New Approach to Solving the Split Common Solution Problem for Monotone Operator Equations in Hilbert Spaces

An Iterative Threshold Algorithm of Log-sum Regularization for Sparse Problem

A Relaxed Inertial Method for Solving Monotone Inclusion Problems with Applications

A Stable Solution of a Nonuniformly Perturbed Quadratic Minimization Problem by the Extragradient Method with Step Size Separated from Zero

Tikhonov Regularization with L^0-Term Complementing a Convex Penalty: L^1 Convergence under Sparsity Constraints

Primal-dual extrapolation methods for monotone inclusions under local Lipschitz continuity

On a Stochastic Differential Equation with Correction Term Governed by a Monotone and Lipschitz Continuous Operator

Modified Tseng Method for Solving Pseudomonotone Variational Inequality Problem in Banach Spaces

The dependence of local regularity of solutions on the summability of coefficients and nonhomogenous term