Abstract:SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 1, Page 212-239, March 2023. We propose a new deterministic Kaczmarz algorithm for solving consistent linear systems [math]. Basically, the algorithm replaces orthogonal projections with reflections in the original scheme of Stefan Kaczmarz. Building on this, we give a geometric description of solutions of linear systems. Suppose [math] is [math], we show that the algorithm generates a series of points distributed with patterns on an [math]-sphere centered on a solution. These points lie evenly on [math] lower-dimensional spheres [math], with the property that for any [math], the midpoint of the centers of [math] is exactly a solution of [math]. With this discovery, we prove that taking the average of [math] points on any [math] effectively approximates a solution up to relative error [math], where [math] characterizes the eigengap of the orthogonal matrix produced by the product of [math] reflections generated by the rows of [math]. We also analyze the connection between [math] and [math], the condition number of [math]. In the worst case [math], while for random matrices [math] on average. Finally, we prove that the algorithm indeed solves the linear system [math], where [math] is the lower-triangular matrix such that [math]. The connection between this linear system and the original one is studied. The numerical tests indicate that this new Kaczmarz algorithm has comparable performance to randomized (block) Kaczmarz algorithms.

On the exponential convergence of the Kaczmarz algorithm

Preasymptotic convergence of randomized Kaczmarz method

Nonlinear Kaczmarz algorithms and their convergence

Randomized Kaczmarz iteration methods: Algorithmic extensions and convergence theory

A Deterministic Kaczmarz Algorithm for Solving Linear Systems

Almost Sure Convergence of the Kaczmarz Algorithm with Random Measurements

On the Randomized Kaczmarz Algorithm

Rates of convergence of randomized Kaczmarz algorithms in Hilbert spaces

Accelerated Kaczmarz Algorithms using History Information

On Multi-Step Randomized Extended Kaczmarz Method for Solving Large Sparse Inconsistent Linear Systems

Convergence rates of the Kaczmarz-Tanabe method for linear systems

Randomized Kaczmarz Algorithm for Inconsistent Linear Systems: An Exact MSE Analysis

A Note on the Randomized Kaczmarz Algorithm for Solving Doubly Noisy Linear Systems

Acceleration of Randomized Kaczmarz Method via the Johnson-Lindenstrauss Lemma

An Accelerated Block Randomized Kaczmarz Method

Randomized Kaczmarz method with adaptive stepsizes for inconsistent linear systems

On Convergence Rates of Kaczmarz-Type Methods with Different Selection Rules of Working Rows

An accelerated randomized Kaczmarz algorithm

Inertial randomized Kaczmarz algorithms for solving coherent linear systems

A quantile-based block Kaczmarz algorithm for solving large consistent linear systems