A Regularization Method for the Proximal Point Algorithm

Hong-Kun Xu

DOI: https://doi.org/10.1007/s10898-006-9002-7

2006-01-01

Journal of Global Optimization

Abstract:A regularization method for the proximal point algorithm of finding a zero for a maximal monotone operator in a Hilbert space is proposed. Strong convergence of this algorithm is proved.

What problem does this paper attempt to address?

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Jinhua Wang,Chong Li,Genaro Lopez,Jen-Chih Yao

DOI: https://doi.org/10.1137/15m1051257

IF: 2.763

2016-01-01

SIAM Journal on Optimization

Abstract:In the present paper, we consider inexact proximal point algorithms for finding singular points of multivalued vector fields on Hadamard manifolds. The rate of convergence is shown to be linear under the mild assumption of metric subregularity. Furthermore, if the sequence of parameters associated with the iterative scheme converges to 0, then the convergence rate is superlinear. At the same time, the finite termination of the inexact proximal point algorithm is also provided under a weak sharp minima-like condition. Applications to optimization problems are provided. Some of our results are new even in Euclidean spaces, while others improve and/or extend some known results in Euclidean spaces. As a matter of fact, in the case of exact proximal point algorithm, our results improve the corresponding results in [G. C. Bento and J. X. Cruz Neto, Opttm., 63 (2014), pp. 1281-1288]. Finally, several examples are provided to illustrate that our results are applicable while the corresponding results in the Hilbert space setting are not.
A Proximal Point Method for the Sum of Maximal Monotone Operators

Hongying Xiao,Xueying Zeng

DOI: https://doi.org/10.1002/mma.3004

2013-01-01

Mathematical Methods in the Applied Sciences

Abstract:In this paper, we concentrate on the maximal inclusion problem of locating the zeros of the sum of maximal monotone operators in the framework of proximal point method. Such problems arise widely in several applied mathematical fields such as signal and image processing. We define two new maximal monotone operators and characterize the solutions of the considered problem via the zeros of the new operators. The maximal monotonicity and resolvent of both of the defined operators are proved and calculated, respectively. The traditional proximal point algorithm can be therefore applied to the considered maximal inclusion problem, and the convergence is ensured. Furthermore, by exploring the relationship between the proposed method and the generalized forward-backward splitting algorithm, we point out that this algorithm is essentially the proximal point algorithm when the operator corresponding to the forward step is the zero operator. Copyright (c) 2013 John Wiley & Sons, Ltd.
A Regularization Method for Treating Zero Points of the Sum of Two Monotone Operators

Xiaolong Qin,Sun Young Cho,Lin Wang

DOI: https://doi.org/10.1186/1687-1812-2014-75

2014-01-01

Fixed Point Theory and Applications

Abstract:In this paper, a regularization method for treating zero points of the sum of two monotone operators is investigated. Strong convergence theorems are established in the framework of Hilbert spaces.
Finite Termination of Inexact Proximal Point Algorithms in Hilbert Spaces

J. H. Wang,C. Li,J.-C. Yao

DOI: https://doi.org/10.1007/s10957-014-0689-1

2014-01-01

Journal of Optimization Theory and Applications

Abstract:In the present paper, we study the finite termination of sequences generated by inexact proximal point algorithms for finding zeroes of a maximal monotone (set-valued) operator T on a Hilbert space. Under some mild conditions, we get that a sequence generated by inexact proximal point algorithm stops after a finite number of iterations. Our results extend the corresponding results in Rockafellar (SIAM J Control Optim 14:877–898, 1976 ). In particular, for optimization problems, our results improve corresponding results in Ferris (Math Progr 50:359–366, 1991 ). As applications, we obtain finite termination of projected gradient method.
Proximal Point Algorithms for Zero Points of Nonlinear Operators

Yuan Qing,Sun Young Cho

DOI: https://doi.org/10.1186/1687-1812-2014-42

2014-01-01

Fixed Point Theory and Applications

Abstract:A proximal point algorithm with double computational errors for treating zero points of accretive operators is investigated. Strong convergence theorems of zero points are established in a Banach space.
The Indefinite Proximal Point Algorithms for Maximal Monotone Operators

Fan Jiang,Xingju Cai,Deren Han

DOI: https://doi.org/10.1080/02331934.2020.1751158

IF: 2.2

2020-01-01

Optimization

Abstract:The proximal point algorithm (PPA) has been widely used in convex optimization. Many algorithms fall into the framework of PPA. To guarantee the convergence of PPA, however, existing results conventionally need to ensure the positive definiteness of the corresponding proximal measure. In some senses, this essentially results in tiny step sizes (or over regularization) for the subproblems and thus inevitably decelerates the overall convergence speed of PPA. In this paper, we investigate the possibility of relaxing the positive definiteness requirement of the proximal measure in PPA. An indefinite PPA for finding a zero of maximal monotone operator is thus proposed via choosing an indefinite proximal regularization term, resulting in larger step sizes. Under some suitable conditions, we prove the global convergence of the proposed algorithm and its extension. To make our method more practical, we suggest to solve the subproblem in an approximate manner and propose two flexible inexact criteria. We show that the condition which guarantees the convergence of the proposed indefinite PPA is tight by a simple example. In addition, we show how to apply the indefinite PPA to some convex models. We report some preliminary numerical results, which show the efficiencies of the proposed algorithms.
Error Sensitivity for Strongly Convergent Modifications of the Proximal Point Algorithm

Yamin Wang,Fenghui Wang,Hong-Kun Xu

DOI: https://doi.org/10.1007/s10957-015-0835-4

2015-01-01

Journal of Optimization Theory and Applications

Abstract:The proximal point algorithm plays an important role in finding zeros of maximal monotone operators. It has however only weak convergence in the infinite-dimensional setting. In the present paper, we provide two contraction-proximal point algorithms. The strong convergence of the two algorithms is proved under two different accuracy criteria on the errors. A new technique of argument is used, and this makes sure that our conditions, which are sufficient for the strong convergence of the algorithms, are weaker than those used by several other authors.
Monotone vector fields and the proximal point algorithm on Hadamard manifolds

C. Li,G. Lopez,V. Martin-Marquez

DOI: https://doi.org/10.1112/jlms/jdn087

2009-01-01

Abstract:The maximal monotonicity notion in Banach spaces is extended to Riemannian manifolds of nonpositive sectional curvature, Hadamard manifolds, and proved to be equivalent to the upper semicontinuity. We consider the problem of finding a singularity of a multivalued vector field in a Hadamard manifold and present a general proximal point method to solve that problem, which extends the known proximal point algorithm in Euclidean spaces. We prove that the sequence generated by our method is well defined and converges to a singularity of a maximal monotone vector field, whenever it exists. Applications in minimization problems with constraints, minimax problems and variational inequality problems, within the framework of Hadamard manifolds, are presented.
A NEW ACCURACY CRITERION FOR THE CONTRACTION PROXIMAL POINT ALGORITHM WITH TWO MONOTONE OPERATORS

Yamin Wang,Hong-Kun Xu

2015-01-01

Journal of nonlinear and convex analysis

Abstract:An iterative algorithm is proposed to find a common zero of two maximal monotone operators A and B in a Hilbert space. This algorithm is a two-step procedure which alternates the operators A and B. The features of this algorithm are its combination of regularization and contraction for the proximal point algorithm, and its strong convergence under different accuracy criteria on the errors.
Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Jinhua Wang,Chong Li,K. F. Ng

DOI: https://doi.org/10.1137/22M152147X

2023-01-01

Abstract:We study the issue of strong convergence of inexact proximal point algorithms (introduced by Rockafellar in [SIAM J. Control Optim., 14 (1976), pp. 877-898]) for maximal monotone operators on Hilbert spaces. A unified global/local strong convergence of inexact proximal point algorithms is established under the Holder metrically subregular condition. Furthermore, quantitative estimates on the convergence rate of inexact proximal point algorithms are also provided. Applying to the special case of the classical (exact) proximal point algorithm, our results improve the corresponding ones in [G. Li and B. S. Mordukhovich, SIAM J. Optim., 22 (2012), pp. 1655-1684]. Finally, as applications, global/local strong convergence and estimates on the convergence rate of inexact proximal point algorithms for optimization problems are presented.
A Strongly Convergent Modification of the Proximal Point Algorithm in Nonsmooth Banach Spaces

Souhail Chebbi,Najla Altwaijry,Hong-Kun Xu

DOI: https://doi.org/10.24193/fpt-ro.2018.1.09

2018-01-01

Fixed Point Theory

Abstract:Rockafellar’s proximal point algorithm is known to be not strongly convergent in general in an infinite-dimensional Hilbert space. Effort has thus been made to modify this algorithm so that strong convergence is guaranteed. In this paper we provide a strongly convergent modification of Rockafellar’s proximal point algorithm in a uniformly convex Banach space which is not necessarily smooth.
On the proximal point algorithms for solving the monotone inclusion problem

Tao Zhang,Shiru Li,Yong Xia

2023-12-22

Abstract:We consider finding a zero point of the maximally monotone operator $T$. First, instead of using the proximal point algorithm (PPA) for this purpose, we employ PPA to solve its Yosida regularization $T_{\lambda}$. Then, based on an $O(a_{k+1})$ ($a_{k+1}\geq \varepsilon>0$) resolvent index of $T$, it turns out that we can establish a convergence rate of $O (1/{\sqrt{\sum_{i=0}^{k}a_{i+1}^2}})$ for both the $\|T_{\lambda}(\cdot)\|$ and the gap function $\mathtt{Gap}(\cdot)$ in the non-ergodic sense, and $O(1/\sum_{i=0}^{k}a_{i+1})$ for $\mathtt{Gap}(\cdot)$ in the ergodic sense. Second, to enhance the convergence rate of the newly-proposed PPA, we introduce an accelerated variant called the Contracting PPA. By utilizing a resolvent index of $T$ bounded by $O(a_{k+1})$ ($a_{k+1}\geq \varepsilon>0$), we establish a convergence rate of $O(1/\sum_{i=0}^{k}a_{i+1})$ for both $\|T_{\lambda}(\cdot)\|$ and $\mathtt {Gap}(\cdot)$, considering the non-ergodic sense. Third, to mitigate the limitation that the Contracting PPA lacks a convergence guarantee, we propose two additional versions of the algorithm. These novel approaches not only ensure guaranteed convergence but also provide sublinear and linear convergence rates for both $\|T_{\lambda}(\cdot)\|$ and $\mathtt {Gap}(\cdot)$, respectively, in the non-ergodic sense.

Optimization and Control
The Developments of Proximal Point Algorithms

Cai Xing-Ju,Guo Ke,Jiang Fan,Wang Kai,Wu Zhong-Ming,Han De-Ren

DOI: https://doi.org/10.1007/s40305-021-00352-x

2022-01-01

Journal of the Operations Research Society of China

Abstract:The problem of finding a zero point of a maximal monotone operator plays a central role in modeling many application problems arising from various fields, and the proximal point algorithm (PPA) is among the fundamental algorithms for solving the zero-finding problem. PPA not only provides a very general framework of analyzing convergence and rate of convergence of many algorithms, but also can be very efficient in solving some structured problems. In this paper, we give a survey on the developments of PPA and its variants, including the recent results with linear proximal term, with the nonlinear proximal term, as well as the inexact forms with various approximate criteria.
Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Hu Yaohua,Li Chong,Meng Kaiwen,Yang Xiaoqi

DOI: https://doi.org/10.1007/s10898-020-00955-3

2020-01-01

Journal of Global Optimization

Abstract:The ℓ _p regularization problem with 0< p< 1 has been widely studied for finding sparse solutions of linear inverse problems and gained successful applications in various mathematics and applied science fields. The proximal gradient algorithm is one of the most popular algorithms for solving the ℓ _p regularisation problem. In the present paper, we investigate the linear convergence issue of one inexact descent method and two inexact proximal gradient algorithms (PGA). For this purpose, an optimality condition theorem is explored to provide the equivalences among a local minimum, second-order optimality condition and second-order growth property of the ℓ _p regularization problem. By virtue of the second-order optimality condition and second-order growth property, we establish the linear convergence properties of the inexact descent method and inexact PGAs under some simple assumptions. Both linear convergence to a local minimal value and linear convergence to a local minimum are provided. Finally, the linear convergence results of these methods are extended to the infinite-dimensional Hilbert spaces. Our results cannot be established under the framework of Kurdyka–Łojasiewicz theory.
An Alternative Regularization Method for Nonexpansive Mappings with Applications

Hong-Kun Xu

DOI: https://doi.org/10.1090/conm/513/10087

2010-01-01

Abstract:We introduce a new regularization method for nonexpansive mappings in a Banach space and prove that if the underlying space is either uniformly smooth or has a weakly continuous duality map, then the regularized solutions converge in norm if and only if the relevant fixed point equation is consistent. We then discretize the regularization method to define iterative algorithms which are proved to be strongly convergent. Moreover, we apply these algorithms to find zeros of maximal monotone and m-accretive operators and solve a quadratic minimization problem.
On the optimal linear convergence factor of the relaxed proximal point algorithm for monotone inclusion problems.

Guoyong Gu,Junfeng Yang

2019-01-01

Abstract: Finding a zero of a maximal monotone operator is fundamental in convex optimization and monotone operator theory, and \emph{proximal point algorithm} (PPA) is a primary method for solving this problem. PPA converges not only globally under fairly mild conditions but also asymptotically at a fast linear rate provided that the underlying inverse operator is Lipschitz continuous at the origin. These nice convergence properties are preserved by a relaxed variant of PPA. Recently, a linear convergence bound was established in [M. Tao, and X. M. Yuan, J. Sci. Comput., 74 (2018), pp. 826-850] for the relaxed PPA, and it was shown that the bound is optimal when the relaxation factor $\gamma$ lies in $[1,2)$. However, for other choices of $\gamma$, the bound obtained by Tao and Yuan is suboptimal. In this paper, we establish tight linear convergence bounds for any choice of $\gamma\in(0,2)$ and make the whole picture about optimal linear convergence bounds clear. These results sharpen our understandings to the asymptotic behavior of the relaxed PPA.
Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Jinhua Wang,Chong Li,Genaro Lopez,Jen-Chih Yao

DOI: https://doi.org/10.1007/s10898-014-0182-2

2014-01-01

Journal of Global Optimization

Abstract:Inexact proximal point methods are extended to find singular points for multivalued vector fields on Hadamard manifolds. Convergence criteria are established under some mild conditions. In particular, in the case of proximal point algorithm, that is, ε _n=0 for each n , our results improve sharply the corresponding results in Li et al. ( 2009 ). Applications to optimization problems, variational inequality problems and gradient methods are also given.
Proximal Point Algorithms for Nonsmooth Convex Optimization with Fixed Point Constraints

Hideaki Iiduka

DOI: https://doi.org/10.48550/arXiv.1602.01932

2016-02-05

Abstract:The problem of minimizing the sum of nonsmooth, convex objective functions defined on a real Hilbert space over the intersection of fixed point sets of nonexpansive mappings, onto which the projections cannot be efficiently computed, is considered. The use of proximal point algorithms that use the proximity operators of the objective functions and incremental optimization techniques is proposed for solving the problem. With the focus on fixed point approximation techniques, two algorithms are devised for solving the problem. One blends an incremental subgradient method, which is a useful algorithm for nonsmooth convex optimization, with a Halpern-type fixed point iteration algorithm. The other is based on an incremental subgradient method and the Krasnosel'ski\uı-Mann fixed point algorithm. It is shown that any weak sequential cluster point of the sequence generated by the Halpern-type algorithm belongs to the solution set of the problem and that there exists a weak sequential cluster point of the sequence generated by the Krasnosel'ski\uı-Mann-type algorithm, which also belongs to the solution set. Numerical comparisons of the two proposed algorithms with existing subgradient methods for concrete nonsmooth convex optimization show that the proposed algorithms achieve faster convergence.

Optimization and Control
Tight Sublinear Convergence Rate of the Proximal Point Algorithm for Maximal Monotone Inclusion Problems

Guoyong Gu,Junfeng Yang

DOI: https://doi.org/10.1137/19m1299049

IF: 2.763

2020-01-01

SIAM Journal on Optimization

Abstract:The tight sublinear convergence rate of the proximal point algorithm for maximal monotone inclusion problems is established based on the squared fixed point residual. By using the performance estimation framework, the tight sublinear convergence rate problem is written as an infinite dimensional nonconvex optimization problem, which is then equivalently reformulated as a finite dimensional semidefinite programming (SDP) problem. By solving the SDP, the exact sublinear rate is computed numerically. Theoretically, by constructing a feasible solution to the dual SDP, an upper bound is obtained for the tight sublinear rate. On the other hand, an example in two dimensional space is constructed to provide a lower bound. The lower bound matches exactly the upper bound obtained from the dual SDP, which also coincides with the numerical rate computed. Hence, we have established the worst case sublinear convergence rate, which is tight in terms of both the order and the constants involved.
A Regularization Algorithm for Zero Points of Accretive Operators

Yuan Qing,Sun Young Cho

DOI: https://doi.org/10.1186/1687-1812-2013-341

2013-01-01

Fixed Point Theory and Applications

Abstract:A regularization algorithm with a computational error for treating accretive operators is investigated. A strong convergence theorem for zero points of accretive operators is established in a reflexive Banach space.

A Regularization Method for the Proximal Point Algorithm

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

A Proximal Point Method for the Sum of Maximal Monotone Operators

A Regularization Method for Treating Zero Points of the Sum of Two Monotone Operators

Finite Termination of Inexact Proximal Point Algorithms in Hilbert Spaces

Proximal Point Algorithms for Zero Points of Nonlinear Operators

The Indefinite Proximal Point Algorithms for Maximal Monotone Operators

Error Sensitivity for Strongly Convergent Modifications of the Proximal Point Algorithm

Monotone vector fields and the proximal point algorithm on Hadamard manifolds

A NEW ACCURACY CRITERION FOR THE CONTRACTION PROXIMAL POINT ALGORITHM WITH TWO MONOTONE OPERATORS

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

A Strongly Convergent Modification of the Proximal Point Algorithm in Nonsmooth Banach Spaces

On the proximal point algorithms for solving the monotone inclusion problem

The Developments of Proximal Point Algorithms

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

An Alternative Regularization Method for Nonexpansive Mappings with Applications

On the optimal linear convergence factor of the relaxed proximal point algorithm for monotone inclusion problems.

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Proximal Point Algorithms for Nonsmooth Convex Optimization with Fixed Point Constraints

Tight Sublinear Convergence Rate of the Proximal Point Algorithm for Maximal Monotone Inclusion Problems

A Regularization Algorithm for Zero Points of Accretive Operators