Abstract:A greedy randomized augmented Kaczmarz (GRAK) method was proposed in Bai and Wu for large and sparse inconsistent linear systems. However, in each iteration of this method, one has to construct two new index sets via computing residual vector with respect to the augmented linear system. Thus, the computational overhead of this method may be large for extremely large‐scale problems. Moreover, there is no reliable stopping criterion for this method. In this work, we are interested in solving large‐scale sparse or dense inconsistent linear systems, and try to enhance the numerical performance of this method. The contributions of this work are as follows. First, we propose an accelerated GRAK method. Theoretical analysis indicates that it converges faster than the GRAK method under a very weak assumption. Second, we apply the semi‐randomized Kaczmarz method to the augmented linear system, and propose a semi‐randomized augmented Kaczmarz method with simple random sampling. In this method, there is no need to access all the information of the data matrix. The convergence of the proposed methods are established. To the best of our knowledge, there are no practical stopping criteria in all randomized Kaczmarz‐type methods till now. To fill‐in this gap, the third contribution of this work is to introduce a practical stopping criterion for Kaczmarz‐type methods, and show its rationality from a theoretical point of view. Numerical results are performed on both real‐world and synthetic data sets, which demonstrate the effectiveness of the proposed stopping criterion, and illustrate that the new methods are often superior to many state‐of‐the‐art Kaczmarz methods for large‐scale inconsistent linear systems.

Strong underrelaxation in Kaczmarz's method for inconsistent systems

Accelerating the distributed Kaczmarz algorithm by strong over-relaxation

The standard form and convergence theory of the relaxation Kaczmarz-Tanabe method for solving linear systems

Choosing relaxation parameter in randomized Kaczmarz method

Randomized Kaczmarz method with adaptive stepsizes for inconsistent linear systems

Remarks on Kaczmarz Algorithm for Solving Consistent and Inconsistent System of Linear Equations

Randomized Kaczmarz Algorithm for Inconsistent Linear Systems: An Exact MSE Analysis

Lax Equivalence for Hyperbolic Relaxation Approximations

Convergence Rate of the Relaxed CQ Algorithm under Hölderian Type Error Bound Property

Randomized Extended Kaczmarz for Solving Least-Squares

On Multi-Step Randomized Extended Kaczmarz Method for Solving Large Sparse Inconsistent Linear Systems

Stability of the Kaczmarz Reconstruction for Stationary Sequences

A Note on the Behavior of the Randomized Kaczmarz Algorithm of Strohmer and Vershynin

A subspace constrained randomized Kaczmarz method for structure or external knowledge exploitation

A Novel Greedy Kaczmarz Method For Solving Consistent Linear Systems

A Semi‐Randomized and Augmented Kaczmarz Method With Simple Random Sampling for Large‐Scale Inconsistent Linear Systems

Convergence Behaviors of Multisplitting Methods with K+1 Relaxed Parameters

On multi-step extended maximum residual Kaczmarz method for solving large inconsistent linear systems

Relaxation strength for multilinear optimization: McCormick strikes back

On a result by Baillon, Bruck, and Reich

Remarks On Hyperbolic Relaxation Systems