Abstract:Solving large-scale systems of nonlinear equations/inequalities is a fundamental problem in computing and optimization. In this paper, we propose a generic successive projection (SP) framework for this problem. The SP sequentially projects the current iterate onto the constraint set corresponding to each nonlinear (in)equality. It extends von Neumann's alternating projection for finding a point in the intersection of two linear subspaces, Bregman's method for finding a common point of convex sets and the Kaczmarz method for solving systems of linear equations to the more general case of multiple nonlinear and nonconvex sets. The existing convergence analyses on randomized Kaczmarz are merely applicable to linear case. There are no theoretical convergence results of the SP for solving nonlinear equations. This paper presents the first proof that the SP locally converges to a solution of nonlinear equations/inequalities at a linear rate. Our work establishes the convergence theory of the SP for the case of multiple nonlinear and nonconvex sets. Besides cyclic and randomized projections, we devise two new greedy projection approaches that significantly accelerate the convergence. Furthermore, the theoretical bounds of the convergence rates are derived. We reveal that the convergence rates are related to the Hoffman constants of the Jacobian matrix of the nonlinear functions at the solution. Applying the SP to solve the graph realization problem, which attracts much attention in theoretical computer science, is discussed.

A Newton-Type Algorithm for Solving Problems of Search Theory.

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

AN ADAPTIVE ANT COLONY SYSTEM ALGORITHM FOR CONTINUOUS-SPACE OPTIMIZATION PROBLEMS

A Surrogate-Based Optimization Algorithm for Network Design Problems

The convergence of conjugate gradient method with nonmonotone line search

Algorithms for the continuous nonlinear resource allocation problem---new implementations and numerical studies

Solving the continuous nonlinear resource allocation problem with an interior point method

Local and Global Behavior for Algorithms of Solving Equations

An Uni ed Convergence Theory for Newton's Type Methods for Zeros of Nonlinear Operators in Banach spaces

A Unified Convergence Theory for Newton-Type Methods for Zeros of Nonlinear Operators in Banach Spaces

A Parallel Method With Hybrid Algorithms For Mixed Integer Nonlinear Programming

A Direct Searching Method for Nonlinear Optimizing Problems and Its Convergence Proof

New Lagrangian dual algorithms for solving the continuous nonlinear resource allocation problem

An Algorithm for Solving the Global Optimization of Nonlinear Integer Programming

Nonlinear Equations Solving with Intelligent Optimization Algorithms: A Survey.

A Networked Parallel Algorithm for Solving Linear Algebraic Equations

Solving Semi-Discrete Optimal Transport Problems: star shapedeness and Newton's method

Efficient Local Search for Nonlinear Real Arithmetic

Successive Projection for Solving Systems of Nonlinear Equations/Inequalities

A new search direction for full-Newton step infeasible interior-point method in linear optimization

Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints