Abstract:Solving large-scale systems of nonlinear equations/inequalities is a fundamental problem in computing and optimization. In this paper, we propose a generic successive projection (SP) framework for this problem. The SP sequentially projects the current iterate onto the constraint set corresponding to each nonlinear (in)equality. It extends von Neumann's alternating projection for finding a point in the intersection of two linear subspaces, Bregman's method for finding a common point of convex sets and the Kaczmarz method for solving systems of linear equations to the more general case of multiple nonlinear and nonconvex sets. The existing convergence analyses on randomized Kaczmarz are merely applicable to linear case. There are no theoretical convergence results of the SP for solving nonlinear equations. This paper presents the first proof that the SP locally converges to a solution of nonlinear equations/inequalities at a linear rate. Our work establishes the convergence theory of the SP for the case of multiple nonlinear and nonconvex sets. Besides cyclic and randomized projections, we devise two new greedy projection approaches that significantly accelerate the convergence. Furthermore, the theoretical bounds of the convergence rates are derived. We reveal that the convergence rates are related to the Hoffman constants of the Jacobian matrix of the nonlinear functions at the solution. Applying the SP to solve the graph realization problem, which attracts much attention in theoretical computer science, is discussed.

Multidimensional Project Algorithm for Solving Linear Equations

Method of Solving Linear Equations Based on Residual Space

A three-term subspace projection method for solving systems of nonlinear monotone equations

A Non-Krylov subspace Method for Solving Large and Sparse Linear System of Equations

Orthogonal projection analysis

A Fast Algorithm for Multidimensional Ellipsoid-Specific Fitting by Minimizing a New Defined Vector Norm of Residuals Using Semidefinite Programming

Successive Projection for Solving Systems of Nonlinear Equations/Inequalities

Variable Projection Algorithms: Theoretical Insights and A Novel Approach for Problems with Large Residual

Nonmonotone variable projection algorithms for matrix decomposition with missing data

A Projection Method Based on Extended Krylov Subspaces for Solving Sylvester Equations

Successive Projection Method for Solving the Unbalanced Procrustes Problem

Computational performance of a projection and rescaling algorithm

Solving Conic Systems via Projection and Rescaling

Proving Inequalities and Solving Global Optimization Problems Via Simplified CAD Projection

A Monotone Projection Algorithm for Solving Fixed Points of Nonlinear Mappings and Equilibrium Problems

Two-Dimensional Matrix Characteristics Method Based on Krylov Subspace and Domain Decomposition Theories

A nonlinear relaxation-strategy-based algorithm for solving sum-of-linear-ratios problems

DFOM algorithm and error analysis for projection methods for solving singular linear system

Adaptive Relaxation Strategy on Basic Iterative Methods for Solving Linear Systems with Single and Multiple Right-Hand Sides

Algorithms for treating equilibrium and fixed point problems

Investigation of highly efficient algorithms for solving linear equations in the discontinuous deformation analysis method