Abstract:We introduce innovative algorithms for computing exact or approximate (minimum-norm) solutions to $Ax=b$ or the {\it normal equation} $A^TAx=A^Tb$, where $A$ is an $m \times n$ real matrix of arbitrary rank. We present more efficient algorithms when $A$ is symmetric PSD. First, we introduce the {\it Triangle Algorithm} (TA), a {\it convex-hull membership} algorithm that given $b_k=Ax_k$ in the ellipsoid $E_{A,\rho}=\{Ax: \Vert x \Vert \leq \rho\}$, it either computes an improved approximation $b_{k+1}=Ax_{k+1}$ or proves $b \not \in E_{A,\rho}$. We then give a dynamic variant of TA, the {\it Centering Triangle Algorithm} (CTA), generating residual, $r_k=b -Ax_k$ via the iteration of $F_1(r)=r-(r^THr/r^TH^2r)Hr$, where $H=AA^T$. If $A$ is symmetric PSD, $H$ can be taken as $A$. Next, for each $t=1, \dots, m$, we derive $F_t(r)=r- \sum_{i=1}^t \alpha_{t,i}(r) H^i r$ whose iterations correspond to a Krylov subspace method with restart. If $\kappa^+(H)$ is the ratio of the largest to smallest positive eigenvalues of $H$, when $Ax=b$ is consistent, in $k=O({\kappa^+(H)}{t^{-1}} \ln \varepsilon^{-1})$ iterations of $F_t$, $\Vert r_k \Vert \leq \varepsilon$. Each iteration takes $O(tN+t^3)$ operations, $N$ the number of nonzero entries in $A$. By directly applying $F_t$ to the normal equation, we get $\Vert A^TAx_k - A^Tb \Vert \leq \varepsilon$ in $O({\kappa^+(AA^T)}{t}^{-1} \ln \varepsilon^{-1})$ iterations. On the other hand, given any residual $r$, we compute $s$, the degree of its minimal polynomial with respect to $H$ in $O(sN+s^3)$ operations. Then $F_s(r)$ gives the minimum-norm solution of $Ax=b$ or an exact solution of $A^TAx=A^Tb$. The proposed algorithms are simple to implementation and theoretically robust. We present sample computational results, comparing the performance of CTA with CG and GMRES methods. The results support CTA as a highly competitive option.

Subproper and Regular Splittings for Restricted Rectangular Linear System

On the Convergence of Subproper (Multi)-Splitting Methods for Solving Rectangular Linear Systems

A Weak Regular Splitting for Singular Linear Systems

Relative Regularity Conditions and Linear Regularity Properties for Split Feasibility Problems in Normed Linear Spaces

On the perturbation and subproper splittings for the generalized inverse AT,S(2) of rectangular matrix A

On the convergence of splittings for semidefinite linear systems

(T,S) Splitting Methods For Computing The Generalized Inverse A((2))T,S And Rectangular Systems

(T, S) splitting methods for computing the generalized inverse and rectangular systems.

Stationary splitting iterative methods for the matrix equation A X B = C

Note to the Mixed-Type Splitting Iterative Method for Z-Matrices Linear Systems

Preconditioned Tensor Splitting AOR Iterative Methods for ℋ‐tensor Equations

A general class of iterative splitting methods for solving linear systems

A Family of Iteration Functions for General Linear Systems

ITERATIVE ALGORITHMS FOR THE LEAST-SQUARES SYMMETRIC SOLUTION OF AXB = C WITH A SUBMATRIX CONSTRAINT

P-REGULAR SPLITTING ITERATIVE METHODS FOR NON-HERMITIAN POSITIVE DEFINITE LINEAR SYSTEMS

On an Iterative Method for Solving the Least Squares Problem of Rank-Deficient Systems

An Efficient Matrix Splitting Method for the Second-Order Cone Complementarity Problem

On Parallel Multisplitting Block Iterative Methods for Linear Systems Arising in the Numerical Solution of Euler Equations.

On parallel multisplitting methods for non-Hermitian positive definite linear systems

MODIFIED SPLITTING METHODS FOR SOLVING NON-HOMOGENOUS MULTI-LINEAR EQUATIONS WITH M-TENSORS

Single-step triangular splitting iteration method for a class of complex symmetric linear system