Abstract:Solving large-scale systems of nonlinear equations/inequalities is a fundamental problem in computing and optimization. In this paper, we propose a generic successive projection (SP) framework for this problem. The SP sequentially projects the current iterate onto the constraint set corresponding to each nonlinear (in)equality. It extends von Neumann's alternating projection for finding a point in the intersection of two linear subspaces, Bregman's method for finding a common point of convex sets and the Kaczmarz method for solving systems of linear equations to the more general case of multiple nonlinear and nonconvex sets. The existing convergence analyses on randomized Kaczmarz are merely applicable to linear case. There are no theoretical convergence results of the SP for solving nonlinear equations. This paper presents the first proof that the SP locally converges to a solution of nonlinear equations/inequalities at a linear rate. Our work establishes the convergence theory of the SP for the case of multiple nonlinear and nonconvex sets. Besides cyclic and randomized projections, we devise two new greedy projection approaches that significantly accelerate the convergence. Furthermore, the theoretical bounds of the convergence rates are derived. We reveal that the convergence rates are related to the Hoffman constants of the Jacobian matrix of the nonlinear functions at the solution. Applying the SP to solve the graph realization problem, which attracts much attention in theoretical computer science, is discussed.

Lanczos Seed Projection Methods for Solving Symmetric Shifted System with Multiple Right-hand Sides

Improved Seed Methods for Symmetric Positive Definite Linear Equations with Multiple Right-hand Sides

Flexible and deflated variants of the block shifted GMRES method

Deflated GMRES for systems with multiple shifts and multiple right-hand sides

Deflated Iterative Methods for Linear Equations with Multiple Right-Hand Sides

On Krylov subspace methods for skew-symmetric and shifted skew-symmetric linear systems

On restarted and deflated block FOM and GMRES methods for sequences of shifted linear systems

A new projection method for solving large Sylvester equations

A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems

Successive Projection for Solving Systems of Nonlinear Equations/Inequalities

A Projection Method Based on Extended Krylov Subspaces for Solving Sylvester Equations

Multidimensional Project Algorithm for Solving Linear Equations

A Non-Krylov subspace Method for Solving Large and Sparse Linear System of Equations

Generalized minimal residual method for systems with multiple right-hand sides

A flexible and adaptive Simpler GMRES with deflated restarting for shifted linear systems

Three double multi-step randomized extended Kaczmarz methods for solving large sparse inconsistent linear systems

On Kaczmarz method with oblique projection for solving large overdetermined linear systems

Method of Solving Linear Equations Based on Residual Space

Solving Conic Systems via Projection and Rescaling

Iterative Methods for Sparse Symmetric Multilinear Systems

The shifted inner-outer iteration methods for solving Sylvester matrix equations