Abstract:The Kaczmarz method is a popular iterative scheme for solving large-scale linear systems. The randomized Kaczmarz method (RK) greatly improves the convergence rate of the Kaczmarz method, by using the rows of the coefficient matrix in random order rather than in their given order. An obvious disadvantage of the randomized Kaczmarz method is its probability criterion for selecting the active or working rows in the coefficient matrix. In [{\sc Z.Z. Bai, W. Wu}, {\em On greedy randomized Kaczmarz method for solving large sparse linear systems}, SIAM Journal on Scientific Computing, 2018, 40: A592--A606], the authors proposed a greedy randomized Kaczmarz method (GRK). However, this method may suffer from heavily computational cost when the size of the matrix is large, and the overhead will be prohibitively large for big data problems. The contribution of this work is as follows. First, from the probability significance point of view, we present a partially randomized Kaczmarz method, which can reduce the computational overhead needed in greedy randomized Kaczmarz method. Second, based on Chebyshev's law of large numbers and Z-test, we apply a simple sampling approach to the partially randomized Kaczmarz method. The convergence of the proposed method is established. Third, we apply the new strategy to the ridge regression problem, and propose a partially randomized Kaczmarz method with simple random sampling for ridge regression. Numerical experiments demonstrate the superiority of the new algorithms over many state-of-the-art randomized Kaczmarz methods for large linear systems problems and ridge regression problems.

Reverse Quantile-RK and its Application to Quantile-RK

Quantile-based Random Kaczmarz for corrupted linear systems of equations

On Subsampled Quantile Randomized Kaczmarz

On Quantile Randomized Kaczmarz for Linear Systems with Time-Varying Noise and Corruption

Quantile-based Random Sparse Kaczmarz for Corrupted, Noisy Linear Inverse Systems

Quantile-based random sparse Kaczmarz for corrupted and noisy linear systems

A quantile-based block Kaczmarz algorithm for solving large consistent linear systems

fastkqr: A Fast Algorithm for Kernel Quantile Regression

A Note on Randomized Kaczmarz Algorithm for Solving Doubly-Noisy Linear Systems

A Note on the Randomized Kaczmarz Algorithm for Solving Doubly Noisy Linear Systems

Communication-Efficient Nonparametric Quantile Regression via Random Features

Optimal Kernel Quantile Learning with Random Features

An accelerated randomized Kaczmarz algorithm

Fast quantile regression in reproducing kernel Hilbert space

Rates of convergence of randomized Kaczmarz algorithms in Hilbert spaces

Approximate nonparametric quantile regression in reproducing kernel Hilbert spaces via random projection

A Kaczmarz Method with Simple Random Sampling for Solving Large Linear Systems

An Accelerated Block Randomized Kaczmarz Method

Randomized Kaczmarz Algorithm for Inconsistent Linear Systems: An Exact MSE Analysis

An Asynchronous Parallel Randomized Kaczmarz Algorithm