Convergence Analysis of the Algorithm in "Efficient and Robust Discrete Conformal Equivalence with Boundary"

Denis Zorin

DOI: https://doi.org/10.48550/arXiv.2109.03436

2021-09-08

Abstract:In this note we prove that the version of Newton algorithm with line search we used in [2] converges quadratically.

Numerical Analysis,Graphics

What problem does this paper attempt to address?

Convergence of Newton'S Method for Systems of Equations with Constant Rank Derivatives

Xiubin Xu,Chong Li

2007-01-01

Journal of Computational Mathematics

Abstract:The convergence properties of Newton's method for systems of equations with constant rank derivatives are studied under the hypothesis that the derivatives satisfy some weak Lipschitz conditions. The unified convergence results, which include Kantorovich type theorems and Smale's point estimate theorems as special cases, are obtained.
Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

C. Li,K.F. Ng

DOI: https://doi.org/10.1016/j.jmaa.2011.11.062

IF: 1.417

2012-01-01

Journal of Mathematical Analysis and Applications

Abstract:Using the convex process theory we study the convergence issues of the iterative sequences generated by the Gauss–Newton method for the convex inclusion problem defined by a cone C and a smooth function F (the derivative is denoted by F′). The restriction in our consideration is minimal and, even in the classical case (the initial point x0 is assumed to satisfy the following two conditions: F′ is Lipschitz around x0 and the convex process Tx0, defined by Tx0⋅=F′(x0)⋅−C, is surjective), our results are new in giving sufficient conditions (which are weaker than the known ones and have a remarkable property being affine-invariant) ensuring the convergence of the iterative sequence with initial point x0. The same study is also made for the so-called convex-composite optimization problem (with objective function given as the composite of a convex function with a smooth map).
On Convergence Of A New Secant-Like Method For Solving Nonlinear Equations

Hongmin Ren,Qingbiao Wu,Weihong Bi

DOI: https://doi.org/10.1016/j.amc.2010.05.092

IF: 4.397

2010-01-01

Applied Mathematics and Computation

Abstract:In this paper, we prove that the order of a new secant-like method presented recently with the so-called order of 2.618 is only 2.414. Some mistakes in the derivation leading to such a conclusion are pointed out. Meanwhile, under the assumption that the second derivative of the involved function is bounded, the convergence radius of the secant-like method is given, and error estimates matching its convergence order are also provided by using a generalized Fibonacci sequence. (C) 2010 Elsevier Inc. All rights reserved.
Convergence Criterion of Newton's Method for Singular Systems with Constant Rank Derivatives

Xiubin Xu,Chong Li

DOI: https://doi.org/10.1016/j.jmaa.2008.04.009

IF: 1.417

2008-01-01

Journal of Mathematical Analysis and Applications

Abstract:The present paper is concerned with the convergence problem of Newton's method to solve singular systems of equations with constant rank derivatives. Under the hypothesis that the derivatives satisfy a type of weak Lipschitz condition, a convergence criterion based on the information around the initial point is established for Newton's method for singular systems of equations with constant rank derivatives. Applications to two special and important cases: the classical Lipschitz condition and the Smale's assumption, are provided; the latter, in particular, extends and improves the corresponding result due to Dedieu and Kim in [J.P. Dedieu, M. Kim, Newton's method for analytic systems of equations with constant rank derivatives, J. Complexity 18 (2002) 187–209].
Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Chong Li,Nuchun Hu,Jinhua Wang

DOI: https://doi.org/10.1016/j.jco.2010.02.001

IF: 1.333

2010-01-01

Journal of Complexity

Abstract:The notions of Lipschitz conditions with L average are introduced to the study of convergence analysis of Gauss–Newton’s method for singular systems of equations. Unified convergence criteria ensuring the convergence of Gauss–Newton’s method for one kind of singular systems of equations with constant rank derivatives are established and unified estimates of radii of convergence balls are also obtained. Applications to some special cases such as the Kantorovich type conditions, γ-conditions and the Smale point estimate theory are provided and some important known results are extended and/or improved.
On Local Behavior of Newton-Type Methods Near Critical Solutions of Constrained Equations

A. F. Izmailov,M. V. Solodov

DOI: https://doi.org/10.1007/s10957-023-02367-1

2024-01-19

Journal of Optimization Theory and Applications

Abstract:For constrained equations with nonisolated solutions and a certain family of Newton-type methods, it was previously shown that if the equation mapping is 2-regular at a given solution with respect to a direction which is interior feasible and which is in the null space of the Jacobian, then there is an associated large (not asymptotically thin) domain of starting points from which the iterates are well defined and converge to the specific solution in question. Under these assumptions, the constrained local Lipschitzian error bound does not hold, unlike the common settings of convergence and rate of convergence analyses. In this work, we complement those previous results by considering the case when the equation mapping is 2-regular with respect to a direction in the null space of the Jacobian which is in the tangent cone to the set, but need not be interior feasible. Under some further conditions, we still show linear convergence of order 1/2 from a large domain around the solution (despite degeneracy, and despite that there may exist other solutions nearby). Our results apply to constrained variants of the Gauss–Newton and Levenberg–Marquardt methods, and to the LP-Newton method. An illustration for a smooth constrained reformulation of the nonlinear complementarity problem is also provided.

mathematics, applied,operations research & management science
Convergence of Newton'S Method and Uniqueness of the Solution of Equations in Banach Spaces Ii

Xing Hua Wang,Chong Li

DOI: https://doi.org/10.1007/s10114-002-0238-y

2003-01-01

Acta Mathematica Sinica English Series

Abstract:Some results on convergence of Newton's method in Banach spaces are established under the assumption that the derivative of the operators satisfies the radius or center Lipschitz condition with a weak L average.
Local Convergence of Inexact Methods under the Hölder Condition

Chong Li,Weiping Shen

DOI: https://doi.org/10.1016/j.cam.2007.12.001

IF: 2.872

2008-01-01

Journal of Computational and Applied Mathematics

Abstract:We Study the convergence properties for some inexact Newton-like methods including the inexact Newton methods for solving nonlinear operator equations on Banach spaces. A new type of residual control is presented. Under the assumption that the derivative of the operator satisfies the Holder condition, the radius of convergence ball of the inexact Newton-like methods with the new type of residual control is estimated, and a linear and/or superlinear convergence property is proved, which extends the corresponding result of [B. Morini, Convergence behaviour of inexact Newton methods, Math. Comput. 68 (1999) 1605-1613]. As an application, we show that the inexact Newton-like method presented in [R.H. Chan, H.L. Chang, S.F. Xu, The inexact Newton-like method for inverse eigenvalue problem, BIT Numer. Math. 43 (2003) 7-20] for solving inverse eigenvalue problems can be regarded equivalently as one of the inexact Newton-like methods considered in this paper. A numerical example is provided to illustrate the convergence performance of the algorithm. (c) 2007 Elsevier B.V. All rights reserved.
The Convergence of Numerical Transfer Schemes in Diffusive Regimes I: Discrete-Ordinate Method

F Golse,S Jin,CD Levermore

DOI: https://doi.org/10.1137/s0036142997315986

IF: 3.039

1999-01-01

SIAM Journal on Numerical Analysis

Abstract:In highly diffusive regimes, the transfer equation with anisotropic boundary conditions has an asymptotic behavior as the mean free path $\epsilon$ tends to zero that is governed by a diffusion equation and boundary conditions obtained through a matched asymptotic boundary layer analysis. A numerical scheme for solving this problem has an $\epsilon^{-1}$ contribution to the truncation error that generally gives rise to a nonuniform consistency with the transfer equation for small $\epsilon$, thus degrading its performance in diffusive regimes. In this paper we show that whenever the discrete-ordinate method has the correct diffusion limit, both in the interior and at the boundaries, its solutions converge to the solution of the transport equation uniformly in $\epsilon$. Our proof of the convergence is based on an asymptotic diffusion expansion and requires error estimates on a matched boundary layer approximation to the solution of the discrete-ordinate method.
A NOTE ON THE CONVERGENCE OF THE DFP ALGORITHM ON QUADRATIC UNIFORMLY CONVEX FUNCTIONS

GH Liu,JY Han,DC Xu

DOI: https://doi.org/10.1080/02331930290019468

IF: 2.2

2002-01-01

Optimization

Abstract:This paper analyzes the open question of the convergence of the DFP algorithm with inexact line searches. We prove that the DFP algorithm is convergent for quadratic uniformly convex function with commonly used inexact line searches.
On the convergence of Newton series and the asymptotics of finite differences

Glenn Bruda

2024-09-02

Abstract:Suppose a complex function $f$ has a Lebesgue measurable inverse Laplace transform. We show that the $n$th order forward and backward differences of $f$ at $z_0\in\mathbb{C}$ tend to zero as $n\to\infty$ whenever $z_0$ lies in the region of absolute convergence of $f$. Under the same hypothesis, we show that the Newton series of $f$ centered at $z_0$ exists and converges in the half-plane $\Re(z)>\Re(z_0)$. Assuming instead that $f$ has a Lebesgue measurable inverse Fourier transform, we show that the $n$th order forward, backward, and central differences of $f$ at any $y\in\mathbb{R}$ are $o(2^n)$. Consequently, we show that the binomial sum $\sum_{k\geq0}{n\choose k}f(k)$ is $o(2^n)$.

Complex Variables,Classical Analysis and ODEs
On the Convergence of Bound Optimization Algorithms

Ruslan R Salakhutdinov,Sam T Roweis,Zoubin Ghahramani

DOI: https://doi.org/10.48550/arXiv.1212.2490

2012-10-19

Abstract:Many practitioners who use the EM algorithm complain that it is sometimes slow. When does this happen, and what can be done about it? In this paper, we study the general class of bound optimization algorithms - including Expectation-Maximization, Iterative Scaling and CCCP - and their relationship to direct optimization algorithms such as gradient-based methods for parameter learning. We derive a general relationship between the updates performed by bound optimization methods and those of gradient and second-order methods and identify analytic conditions under which bound optimization algorithms exhibit quasi-Newton behavior, and conditions under which they possess poor, first-order convergence. Based on this analysis, we consider several specific algorithms, interpret and analyze their convergence properties and provide some recipes for preprocessing input to these algorithms to yield faster convergence behavior. We report empirical results supporting our analysis and showing that simple data preprocessing can result in dramatically improved performance of bound optimizers in practice.

Machine Learning
Fast convergence to non-isolated minima: four equivalent conditions for $\mathrm{C}^2$ functions

Quentin Rebjock,Nicolas Boumal

2023-03-01

Abstract:Optimization algorithms can see their local convergence rates deteriorate when the Hessian at the optimum is singular. These singularities are inescapable when the optima are non-isolated. Yet, under the right circumstances, several algorithms preserve their favorable rates even when optima form a continuum (e.g., due to over-parameterization). This has been explained under various structural assumptions, including the Polyak--Łojasiewicz inequality, Quadratic Growth and the Error Bound. We show that, for cost functions which are twice continuously differentiable ($\mathrm{C}^2$), those three (local) properties are equivalent. Moreover, we show they are equivalent to the Morse--Bott property, that is, local minima form differentiable submanifolds, and the Hessian of the cost function is positive definite along its normal directions. We leverage this insight to improve local convergence guarantees for safe-guarded Newton-type methods under any (hence all) of the above assumptions. First, for adaptive cubic regularization, we secure quadratic convergence even with inexact subproblem solvers. Second, for trust-region methods, we argue convergence can fail with an exact subproblem solver, then proceed to show linear convergence with an inexact one (Cauchy steps).

Optimization and Control
On the Convergence of Interior-Point Methods for Bound-Constrained Nonlinear Optimization Problems with Noise

Shima Dezfulian,Andreas Wächter

2024-05-19

Abstract:We analyze the convergence properties of a modified barrier method for solving bound-constrained optimization problems where evaluations of the objective function and its derivatives are affected by bounded and non-diminishing noise. The only modification compared to a standard barrier method is a relaxation of the Armijo line-search condition. We prove that the algorithm generates iterates at which the size of the barrier function gradient eventually falls below a threshold that converges to zero if the noise level converges to zero. Based on this result, we propose a practical stopping test that does not require estimates of unknown problem parameters and identifies iterations in which the theoretical threshold is reached. We also analyze the local convergence properties of the method when noisy second derivatives are used. Under a strict-complementarity assumption, we show that iterates stay in a neighborhood around the optimal solution once it is entered. The neighborhood is defined in a scaled norm that becomes narrower for variables with active bound constraints as the barrier parameter is decreased. As a consequence, we show that active bound constraints can be identified despite noise. Numerical results demonstrate the effectiveness of the stopping test and illustrate the active-set identification properties of the method.

Optimization and Control
Representation of conformal maps by rational functions

Abinand Gopal,Lloyd N. Trefethen

DOI: https://doi.org/10.1007/s00211-019-01023-z

IF: 2.5

2019-02-02

Numerische Mathematik

Abstract:The traditional view in numerical conformal mapping is that once the boundary correspondence function has been found, the map and its inverse can be evaluated by contour integrals. We propose that it is much simpler, and 10–1000 times faster, to represent the maps by rational functions computed by the AAA algorithm. To justify this claim, first we prove a theorem establishing root-exponential convergence of rational approximations near corners in a conformal map, generalizing a result of D. J. Newman in 1964. This leads to the new algorithm for approximating conformal maps of polygons. Then we turn to smooth domains and prove a sequence of four theorems establishing that in any conformal map of the unit circle onto a region with a long and slender part, there must be a singularity or loss of univalence exponentially close to the boundary, and polynomial approximations cannot be accurate unless of exponentially high degree. This motivates the application of the new algorithm to smooth domains, where it is again found to be highly effective.

mathematics, applied
Convergence Analysis of Discrete Conformal Transformation

Zhenyue Zhang,Zhong-Heng Tan

DOI: https://doi.org/10.48550/arXiv.2210.10990

2022-10-20

Numerical Analysis

Abstract:Continuous conformal transformation minimizes the conformal energy. The convergence of minimizing discrete conformal energy when the discrete mesh size tends to zero is an open problem. This paper addresses this problem via a careful error analysis of the discrete conformal energy. Under a weak condition on triangulation, the discrete function minimizing the discrete conformal energy converges to the continuous conformal mapping as the mesh size tends to zero.
Newton-like method with some remarks

Xinyuan Wu

DOI: https://doi.org/10.1016/j.amc.2006.10.003

IF: 4.397

2007-01-01

Applied Mathematics and Computation

Abstract:This paper presents the new convergence analysis for the Newton-like method proposed by [Xinyuan Wu, A new continuation Newton-like method and its deformation, Appl. Math. Comput. 112 (2000) 75–78.] Compared with the original version of the convergence analysis, the restriction imposed on f″(x) is removed thoroughly. In order to guarantee the quadratic convergence of the Newton-like method, it is suffices to suppose that f′(x∗)≠0 and f′(x) is local Lipschitz near x∗, where x∗ is a solution of nonlinear equation f(x)=0. Moreover, some comments are given with examples for the Newton-like method, in comparison with Newton’s method. It can be concluded that the new algorithm is more feasible, effective than Newton’s method. In particularly, when it happens that, the derivative of the function f(x) at an iterate is singular or almost singular, the Newton-like method is vast superior to the classical Newton method. The numerical results of the paper strongly support the conclusion.
A semi-local convergence theorem for a robust revised Newton's method

Zhengyu Wang,Xinyuan Wu

DOI: https://doi.org/10.1016/j.camwa.2009.05.022

IF: 3.218

2009-01-01

Computers & Mathematics with Applications

Abstract:It is well known that Newton's iteration will abort due to the overflow if the derivative of the function at an iterate is singular or almost singular. In this paper, we study a robust revised Newton's method for solving nonlinear equations, which can be carried out with a starting point with a degenerate derivative at an iterative step. It is proved that the method is convergent under the conditions of the Newton-Kantorovich theorem, which implies a larger convergence domain of the method. We also show that our method inherits the fast convergence of Newton's method. Numerical experiments are performed to show the robustness of the proposed method in comparison with the standard Newton's method.
A Convergence Theorem for the Newton-Like Methods under Some Kind of Weak Lipschitz Conditions

Min Wu

DOI: https://doi.org/10.1016/j.jmaa.2007.07.073

IF: 1.417

2008-01-01

Journal of Mathematical Analysis and Applications

Abstract:We provide sufficient conditions for the convergence of the Newton-like methods in the assumption that the derivative satisfies some kind of weak Lipschitz conditions. Consequently, some important convergence theorems follow from our main result in this paper.
On the Convergence Rate of a Quasi-Newton Method for Inverse Eigenvalue Problems

RH Chan,SF Xu,HM Zhou

DOI: https://doi.org/10.1137/s0036142997327051

IF: 3.039

1999-01-01

SIAM Journal on Numerical Analysis

Abstract:In this paper, we first note that the proof of the quadratic convergence of the quasi-Newton method as given in Friedland, Nocedal, and Overton [ SIAM J. Numer. Anal., 24 (1987), pp. 634--667] is incorrect. Then we give a correct proof of the convergence.

Convergence Analysis of the Algorithm in "Efficient and Robust Discrete Conformal Equivalence with Boundary"

Convergence of Newton'S Method for Systems of Equations with Constant Rank Derivatives

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

On Convergence Of A New Secant-Like Method For Solving Nonlinear Equations

Convergence Criterion of Newton's Method for Singular Systems with Constant Rank Derivatives

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

On Local Behavior of Newton-Type Methods Near Critical Solutions of Constrained Equations

Convergence of Newton'S Method and Uniqueness of the Solution of Equations in Banach Spaces Ii

Local Convergence of Inexact Methods under the Hölder Condition

The Convergence of Numerical Transfer Schemes in Diffusive Regimes I: Discrete-Ordinate Method

A NOTE ON THE CONVERGENCE OF THE DFP ALGORITHM ON QUADRATIC UNIFORMLY CONVEX FUNCTIONS

On the convergence of Newton series and the asymptotics of finite differences

On the Convergence of Bound Optimization Algorithms

Fast convergence to non-isolated minima: four equivalent conditions for $\mathrm{C}^2$ functions

On the Convergence of Interior-Point Methods for Bound-Constrained Nonlinear Optimization Problems with Noise

Representation of conformal maps by rational functions

Convergence Analysis of Discrete Conformal Transformation

Newton-like method with some remarks

A semi-local convergence theorem for a robust revised Newton's method

A Convergence Theorem for the Newton-Like Methods under Some Kind of Weak Lipschitz Conditions

On the Convergence Rate of a Quasi-Newton Method for Inverse Eigenvalue Problems