On Convergence of the Gauss-Newton Method for Convex Composite Optimization

Chong Li,Xinghua Wang

DOI: https://doi.org/10.1007/s101070100249

IF: 2.7

2002-01-01

Mathematical Programming

Abstract:The local quadratic convergence of the Gauss-Newton method for convex composite optimization f=h∘F is established for any convex function h with the minima set C, extending Burke and Ferris’ results in the case when C is a set of weak sharp minima for h.

What problem does this paper attempt to address?

Gauss-Newton Methods for A Class of Nonsmooth Optimization Problems

C Li,XH Wang

2000-01-01

Progress in Natural Science

Abstract:The local quadratic convergence of the Gauss-Newton method for convex composite optimizations is established for any convex function with a minima set. This work extends Burke and Ferris' results when this minima set is a set of weak sharp minima for the convex function.
Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

C. Li,K.F. Ng

DOI: https://doi.org/10.1016/j.jmaa.2011.11.062

IF: 1.417

2012-01-01

Journal of Mathematical Analysis and Applications

Abstract:Using the convex process theory we study the convergence issues of the iterative sequences generated by the Gauss–Newton method for the convex inclusion problem defined by a cone C and a smooth function F (the derivative is denoted by F′). The restriction in our consideration is minimal and, even in the classical case (the initial point x0 is assumed to satisfy the following two conditions: F′ is Lipschitz around x0 and the convex process Tx0, defined by Tx0⋅=F′(x0)⋅−C, is surjective), our results are new in giving sufficient conditions (which are weaker than the known ones and have a remarkable property being affine-invariant) ensuring the convergence of the iterative sequence with initial point x0. The same study is also made for the so-called convex-composite optimization problem (with objective function given as the composite of a convex function with a smooth map).
Majorizing Functions and Convergence of the Gauss-Newton Method for Convex Composite Optimization

Chong Li,K. F. Ng

DOI: https://doi.org/10.1137/06065622x

IF: 2.763

2007-01-01

SIAM Journal on Optimization

Abstract:We introduce a notion of quasi regularity for points with respect to the inclusion F( x). C, where F is a nonlinear Frechet differentiable function from R-v to R-m. When C is the set of minimum points of a convex real- valued function h on Rm and F ' satisfies the L- average Lipschitz condition of Wang, we use the majorizing function technique to establish the semilocal linear/ quadratic convergence of sequences generated by the Gauss - Newton method ( with quasiregular initial points) for the convex composite function h. F. Results are new even when the initial point is regular and F ' is Lipschitz.
Gauss–Newton Method for Convex Composite Optimizations on Riemannian Manifolds

Jin-Hua Wang,Jen-Chih Yao,Chong Li

DOI: https://doi.org/10.1007/s10898-010-9638-1

2011-01-01

Journal of Global Optimization

Abstract:A notion of quasi-regularity is extended for the inclusion problem ${F(p)\in C}$ , where F is a differentiable mapping from a Riemannian manifold M to ${\mathbb R^n}$ . When C is the set of minimum points of a convex real-valued function h on ${\mathbb R^n}$ and DF satisfies the L-average Lipschitz condition, we use the majorizing function technique to establish the semi-local convergence of sequences generated by the Gauss-Newton method (with quasi-regular initial points) for the convex composite function h ◦ F on Riemannian manifold. Two applications are provided: one is for the case of regularities on Riemannian manifolds and the other is for the case when C is a cone and DF(p 0)(·) − C is surjective. In particular, the results obtained in this paper extend the corresponding one in Wang et al. (Taiwanese J Math 13:633–656, 2009).
Convergence and Uniqueness Properties of Gauss-Newton's Method

C Li,WH Zhang,XQ Jin

DOI: https://doi.org/10.1016/s0898-1221(04)90086-7

IF: 3.218

2004-01-01

Computers & Mathematics with Applications

Abstract:The generalized radius and center Lipschitz conditions with L average are introduced to investigate the convergence of Gauss-Newton's method for finding the nonlinear least squares solution of nonlinear equations. The radii of the convergence ball of Gauss-Newton's method and the uniqueness ball of the solution are estimated. Some applications are given.
Convergence of the Gauss-Newton method for convex composite optimization problems under majorant condition on Riemannian manifolds

Qamrul Hasan Ansari,Moin Uddin,Jen-Chih Yao

DOI: https://doi.org/10.1016/j.jco.2023.101788

IF: 1.333

2024-02-01

Journal of Complexity

Abstract:In this paper, we consider convex composite optimization problems on Riemannian manifolds, and discuss the semi-local convergence of the Gauss-Newton method with quasi-regular initial point and under the majorant condition. As special cases, we also discuss the convergence of the sequence generated by the Gauss-Newton method under Lipschitz-type condition, or under γ-condition.

mathematics, applied
On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Yaohua Hu,Chong Li,Xiaoqi Yang

DOI: https://doi.org/10.1137/140993090

IF: 2.763

2016-01-01

SIAM Journal on Optimization

Abstract:In the present paper, we investigate a linearized proximal algorithm (LPA) for solving a convex composite optimization problem.Each iteration of the LPA is a proximal minimization of the convex composite function with the inner function being linearizedat the current iterate. The LPA has the attractive computational advantage that the solution of each subproblem is a singleton,which avoids the difficulty as in the Gauss--Newton method (GNM) of finding a solution with minimum norm among the set of minimaof its subproblem, while still maintaining the same local convergence rate as that of the GNM. Under the assumptions of localweak sharp minima of order $p$ ($p \in [1,2]$) and a quasi-regularity condition, we establish a local superlinear convergencerate for the LPA. We also propose a globalization strategy for the LPA based on a backtracking line-search and an inexact versionof the LPA. We further apply the LPA to solve a (possibly nonconvex) feasibility problem, as well as a sensor network localizationproblem. Our numerical results illustrate that the LPA meets the demand for an efficient and robust algorithm for the sensor networklocalization problem.
Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Chong Li,Nuchun Hu,Jinhua Wang

DOI: https://doi.org/10.1016/j.jco.2010.02.001

IF: 1.333

2010-01-01

Journal of Complexity

Abstract:The notions of Lipschitz conditions with L average are introduced to the study of convergence analysis of Gauss–Newton’s method for singular systems of equations. Unified convergence criteria ensuring the convergence of Gauss–Newton’s method for one kind of singular systems of equations with constant rank derivatives are established and unified estimates of radii of convergence balls are also obtained. Applications to some special cases such as the Kantorovich type conditions, γ-conditions and the Smale point estimate theory are provided and some important known results are extended and/or improved.
On the local convergence of the semismooth Newton method for composite optimization

Jinglu Hu,Tonghua Tian,Shaohua Pan,Zaiwen Wen

DOI: https://doi.org/10.48550/arxiv.2211.01127

2022-01-01

Abstract:In this paper, we consider a large class of nonlinear equations derived from first-order type methods for solving composite optimization problems. Traditional approaches to establishing superlinear convergence rates of semismooth Newton-type methods for solving nonlinear equations usually postulate either nonsingularity of the B-Jacobian or smoothness of the equation. We investigate the feasibility of both conditions. For the nonsingularity condition, we present equivalent characterizations in broad generality, and illustrate that they are easy-to-check criteria for some examples. For the smoothness condition, we show that it holds locally for a large class of residual mappings derived from composite optimization problems. Furthermore, we investigate a relaxed version of the smoothness condition - smoothness restricted to certain active manifolds. We present a conceptual algorithm utilizing such structures and prove that it has a superlinear convergence rate.
Convergence Analysis of the Gauss–Newton-Type Method for Lipschitz-Like Mappings

M. H. Rashid,S. H. Yu,C. Li,S. Y. Wu

DOI: https://doi.org/10.1007/s10957-012-0206-3

2012-01-01

Abstract:We introduce in the present paper a Gauss–Newton-type method for solving generalized equations defined by sums of differentiable mappings and set-valued mappings in Banach spaces. Semi-local convergence and local convergence of the Gauss–Newton-type method are analyzed.
A linearly convergent Gauss-Newton subgradient method for ill-conditioned problems

Damek Davis,Tao Jiang

DOI: https://doi.org/10.48550/arXiv.2212.13278

2022-12-27

Abstract:We analyze a preconditioned subgradient method for optimizing composite functions $h \circ c$, where $h$ is a locally Lipschitz function and $c$ is a smooth nonlinear mapping. We prove that when $c$ satisfies a constant rank property and $h$ is semismooth and sharp on the image of $c$, the method converges linearly. In contrast to standard subgradient methods, its oracle complexity is invariant under reparameterizations of $c$.

Optimization and Control
Extended Newton Methods for Multiobjective Optimization: Majorizing Function Technique and Convergence Analysis

Jinhua Wang,Yaohua Hu,Carisa Kwok Wai Yu,Chong Li,Xiaoqi Yang

DOI: https://doi.org/10.1137/18m1191737

IF: 2.763

2019-01-01

SIAM Journal on Optimization

Abstract:We consider the extended Newton method for approaching a Pareto optimum of a multiobjective optimization problem, establish quadratic convergence criteria, and estimate a radius of convergence ball under the assumption that the Hessians of objective functions satisfy an L-average Lipschitz condition. These convergence theorems significantly improve the corresponding ones in [J. Fliege, L. M. G. Drummond, and B. F. Svaiter, SIAM J. Optim., 20 (2009), pp. 602-626]. As applications of the obtained results, convergence theorems under the classical Lipschitz condition or the gamma-condition are presented for multiobjective optimization, and the global quadratic convergence results of the extended Newton method with Armijo/Goldstein/Wolfe line-search schemes are also provided.
Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

X. M. Wang,J. H. Wang,C. Li

DOI: https://doi.org/10.1007/s10957-023-02235-y

2023-01-01

Abstract:We study the issue of convergence for inexact steepest descent algorithm (employing general step sizes) for multiobjective optimizations on general Riemannian manifolds (without curvature constraints). Under the assumption of the local convexity/quasi-convexity, local/global convergence results are established. Furthermore, without the assumption of the local convexity/quasi-convexity, but under an error bound-like condition, local/global convergence results and convergence rate estimates are presented, which are new even in the linear space setting. Our results improve/extend the corresponding ones in (Wang et al. in SIAM J Optim 31(1):172–199, 2021) for scalar optimization problems on Riemannian manifolds to multiobjective ones. Finally, for the special case when the inexact steepest descent algorithm employing Armijo rule, our results improve/extend the corresponding ones in (Ferreira et al. in J Optim Theory Appl 184:507–533, 2020) by relaxing curvature constraints.
Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Jinhua Wang,Xiangmei Wang,Chong Li,Jen-Chih Yao

DOI: https://doi.org/10.1137/19m1289285

IF: 2.763

2021-01-01

SIAM Journal on Optimization

Abstract:We study the convergence issue for the gradient algorithm (employing general step sizes) for optimization problems on general Riemannian manifolds (without curvature constraints). Under the assumption of the local convexity/quasi-convexity (resp. weak sharp minima), local/global convergence (resp. linear convergence) results are established. As an application, the linear convergence properties of the gradient algorithm employing the constant step sizes and the Armijo step sizes for finding the Riemannian $L^p$ ($p\in[1,+\infty)$) centers of mass are explored, respectively, which in particular extend and/or improve the corresponding results in \cite{Afsari2013}.
Convergence Analysis of a Method for Variational Inclusions

M. H. Rashid,J. H. Wang,C. Li

DOI: https://doi.org/10.1080/00036811.2011.618127

2011-01-01

Applicable Analysis

Abstract:Consider the following variational inclusion problem: where f is differentiable in a neighbourhood of a solution and g is differentiable at , and F is a set-valued mapping, and the method introduced in Jean-Alexis and Pietrus [C. Jean-Alexis and A. Pietrus, On the convergence of some methods for variational inclusions, Rev. R. Acad. Cien. serie A. Mat. 102(2) (2008), pp. 355–361] for solving this problem: where ∇f(x) denotes the Fréchet derivative of f at x and [x, y; g] the first-order divided difference of g on the points x and y. Local converge analysis are provided for the method under the weaker conditions than Jean-Alexis and Pietrus (2008). Moreover, if ∇f and the first-order divided difference of g are p-Hölder continuous at a solution, then we show that this method converges superlinearly. In particular, our results extend the corresponding ones Jean-Alexis and Pietrus (2008), and fix a gap in the proof in (Jean-Alexis and Pietrus (2008), Theorem 1).
Convergence of Newton'S Method and Uniqueness of the Solution of Equations in Banach Spaces Ii

Xing Hua Wang,Chong Li

DOI: https://doi.org/10.1007/s10114-002-0238-y

2003-01-01

Acta Mathematica Sinica English Series

Abstract:Some results on convergence of Newton's method in Banach spaces are established under the assumption that the derivative of the operators satisfies the radius or center Lipschitz condition with a weak L average.
Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

Xiangmei Wang,Chong Li,Jinhua Wang,Jen-Chih Yao

DOI: https://doi.org/10.1137/14099961x

IF: 2.763

2015-01-01

SIAM Journal on Optimization

Abstract:We study the convergence issue of the subgradient algorithm for solving the convex feasibility problems in Riemannian manifolds, which was first proposed and analyzed by Bento and Melo [J. Optim. Theory Appl., 152 (2012), pp. 773-785]. The linear convergence property about the subgradient algorithm for solving the convex feasibility problems with the Slater condition in Riemannian manifolds are established, and some step sizes rules are suggested for finite convergence purposes, which are motivated by the work due to De Pierro Iusem [Appl. Math. Optim., 17 (1988), pp. 225-235]. As a by-product, the convergence result of this algorithm is obtained for the convex feasibility problem without the Slater condition assumption. These results extend and/or improve the corresponding known ones in both the Euclidean space and Riemannian manifolds.
On the Local Convergence of a Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization

Milzarek Andre,Xiao Xiantao,Wen Zaiwen,Ulbrich Michael

DOI: https://doi.org/10.1007/s11425-020-1865-1

2022-01-01

Science China Mathematics

Abstract:In this work, we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth convex terms in the objective function. We assume that the gradient and Hessian information of the smooth part of the objective function can only be approximated and accessed via calling stochastic firstand second-order oracles. The approach combines stochastic semismooth Newton steps, stochastic proximal gradient steps and a globalization strategy based on growth conditions. We present tail bounds and matrix concentration inequalities for the stochastic oracles that can be utilized to control the approximation errors via appropriately adjusting or increasing the sampling rates. Under standard local assumptions, we prove that the proposed algorithm locally turns into a pure stochastic semismooth Newton method and converges r-linearly or r-superlinearly with high probability.
Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Jinhua Wang,Chong Li,Genaro Lopez,Jen-Chih Yao

DOI: https://doi.org/10.1007/s10898-014-0182-2

2014-01-01

Journal of Global Optimization

Abstract:Inexact proximal point methods are extended to find singular points for multivalued vector fields on Hadamard manifolds. Convergence criteria are established under some mild conditions. In particular, in the case of proximal point algorithm, that is, ε _n=0 for each n , our results improve sharply the corresponding results in Li et al. ( 2009 ). Applications to optimization problems, variational inequality problems and gradient methods are also given.

On Convergence of the Gauss-Newton Method for Convex Composite Optimization

Gauss-Newton Methods for A Class of Nonsmooth Optimization Problems

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

Majorizing Functions and Convergence of the Gauss-Newton Method for Convex Composite Optimization

Gauss–Newton Method for Convex Composite Optimizations on Riemannian Manifolds

Convergence and Uniqueness Properties of Gauss-Newton's Method

Convergence of the Gauss-Newton method for convex composite optimization problems under majorant condition on Riemannian manifolds

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

On the local convergence of the semismooth Newton method for composite optimization

Convergence Analysis of the Gauss–Newton-Type Method for Lipschitz-Like Mappings

A linearly convergent Gauss-Newton subgradient method for ill-conditioned problems

Extended Newton Methods for Multiobjective Optimization: Majorizing Function Technique and Convergence Analysis

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Convergence Analysis of a Method for Variational Inclusions

Convergence of Newton'S Method and Uniqueness of the Solution of Equations in Banach Spaces Ii

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

On the Local Convergence of a Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds