Abstract:In this paper, we consider a nonlinear least squares model for the phase retrieval problem. Since the Hessian matrix may not be positive definite and the Gauss-Newton (GN) matrix is singular at any optimal solution, we propose a modified Levenberg-Marquardt (LM) method, where the Hessian is substituted by a summation of the GN matrix and a regularization term. Similar to the well-known Wirtinger flow (WF) algorithm under certain assumptions, we start from an initial point provably close to the set of the global optimal solutions. Global linear convergence and local quadratic convergence to the global solution set are established by estimating the smallest nonzero eigenvalues of the GN matrix, establishing local error bound properties and constructing a modified regularization condition. The computational cost becomes tractable if a preconditioned conjugate gradient method is applied to solve the LM equation inexactly. Specifically, the pre-conditioner is constructed from the expectation of the LM coefficient matrix by assuming the independence between the measurements and iteration point. The initialization can be dropped by slightly modifying the algorithm. Preliminary numerical experiments show that our algorithm is robust and it is often faster than the WF method on both random examples and natural image recovery.

An Adaptive Multi-step Levenberg–Marquardt Method

A New Adaptive Levenberg–Marquardt Method for Nonlinear Equations and Its Convergence Rate under the Hölderian Local Error Bound Condition

An accelerated adaptive two-step Levenberg–Marquardt method with the modified Metropolis criterion

An Enhanced Levenberg--Marquardt Method via Gram Reduction

The convergence of conjugate gradient method with nonmonotone line search

Two-Step Fifth-Order Efficient Jacobian-Free Iterative Method for Solving Nonlinear Systems

A two-step Broyden-like method for nonlinear equations

Adaptive Levenberg–Marquardt Algorithm: A New Optimization Strategy for Levenberg–Marquardt Neural Networks

The complex-step Newton method and its convergence

The Levenberg–Marquardt method: an overview of modern convergence theories and more

A Ulm-like Method for Inverse Eigenvalue Problems

On inertial Levenberg-Marquardt type methods for solving nonlinear ill-posed operator equations

A New Fast Adaptive Linearized Alternating Direction Multiplier Method for Convex Optimization

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes

Riemannian Levenberg-Marquardt Method with Global and Local Convergence Properties

Incremental Gauss--Newton Methods with Superlinear Convergence Rates

Globally Convergent Levenberg-Marquardt Method for Phase Retrieval.

A Smoothing Levenberg-Marquardt Type Method for Lcp

A structured diagonal Hessian approximation method with evaluation complexity analysis for nonlinear least squares

An adaptive linearized alternating direction multiplier method with a relaxation step for convex programming

A Matrix-Free Newton Method