Abstract:The Kaczmarz method is a popular iterative scheme for solving large-scale linear systems. The randomized Kaczmarz method (RK) greatly improves the convergence rate of the Kaczmarz method, by using the rows of the coefficient matrix in random order rather than in their given order. An obvious disadvantage of the randomized Kaczmarz method is its probability criterion for selecting the active or working rows in the coefficient matrix. In [{\sc Z.Z. Bai, W. Wu}, {\em On greedy randomized Kaczmarz method for solving large sparse linear systems}, SIAM Journal on Scientific Computing, 2018, 40: A592--A606], the authors proposed a greedy randomized Kaczmarz method (GRK). However, this method may suffer from heavily computational cost when the size of the matrix is large, and the overhead will be prohibitively large for big data problems. The contribution of this work is as follows. First, from the probability significance point of view, we present a partially randomized Kaczmarz method, which can reduce the computational overhead needed in greedy randomized Kaczmarz method. Second, based on Chebyshev's law of large numbers and Z-test, we apply a simple sampling approach to the partially randomized Kaczmarz method. The convergence of the proposed method is established. Third, we apply the new strategy to the ridge regression problem, and propose a partially randomized Kaczmarz method with simple random sampling for ridge regression. Numerical experiments demonstrate the superiority of the new algorithms over many state-of-the-art randomized Kaczmarz methods for large linear systems problems and ridge regression problems.

Extension of Sparse Randomized Kaczmarz Algorithm for Multiple Measurement Vectors

Sparse Randomized Kaczmarz for Support Recovery of Jointly Sparse Corrupted Multiple Measurement Vectors

Stochastic Greedy Algorithms For Multiple Measurement Vectors

A Weighted Randomized Sparse Kaczmarz Method for Solving Linear Systems

Three double multi-step randomized extended Kaczmarz methods for solving large sparse inconsistent linear systems

Faster randomized block sparse Kaczmarz by averaging

On Multi-Step Randomized Extended Kaczmarz Method for Solving Large Sparse Inconsistent Linear Systems

Randomized Kaczmarz algorithm with averaging and block projection

Randomized regularized extended Kaczmarz algorithms for tensor recovery

Randomized Kaczmarz Algorithm for Inconsistent Linear Systems: An Exact MSE Analysis

Randomized Kaczmarz for tensor linear systems

Signal recovery for jointly sparse vectors with different sensing matrices.

Randomized block Kaczmarz method with projection for solving least squares

A Kaczmarz Method with Simple Random Sampling for Solving Large Linear Systems

Parallelization Strategies for the Randomized Kaczmarz Algorithm on Large-Scale Dense Systems

Sparse Sampling Kaczmarz-Motzkin Method with Linear Convergence

A randomized sparse Kaczmarz solver for sparse signal recovery via minimax‐concave penalty

Novel Algorithm for Sparse Solutions to Linear Inverse Problems with Multiple Measurements

Randomized double and triple Kaczmarz for solving extended normal equations

Efficient High-Dimensional Inference in the Multiple Measurement Vector Problem

A subspace constrained randomized Kaczmarz method for structure or external knowledge exploitation