Abstract:SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 1, Page 212-239, March 2023. We propose a new deterministic Kaczmarz algorithm for solving consistent linear systems [math]. Basically, the algorithm replaces orthogonal projections with reflections in the original scheme of Stefan Kaczmarz. Building on this, we give a geometric description of solutions of linear systems. Suppose [math] is [math], we show that the algorithm generates a series of points distributed with patterns on an [math]-sphere centered on a solution. These points lie evenly on [math] lower-dimensional spheres [math], with the property that for any [math], the midpoint of the centers of [math] is exactly a solution of [math]. With this discovery, we prove that taking the average of [math] points on any [math] effectively approximates a solution up to relative error [math], where [math] characterizes the eigengap of the orthogonal matrix produced by the product of [math] reflections generated by the rows of [math]. We also analyze the connection between [math] and [math], the condition number of [math]. In the worst case [math], while for random matrices [math] on average. Finally, we prove that the algorithm indeed solves the linear system [math], where [math] is the lower-triangular matrix such that [math]. The connection between this linear system and the original one is studied. The numerical tests indicate that this new Kaczmarz algorithm has comparable performance to randomized (block) Kaczmarz algorithms.

The randomized circumcentered-reflection iteration method for solving consistent linear equations

Randomized Iterative Methods for Linear Systems

A surrogate hyperplane Kaczmarz method for solving consistent linear equations

A Deterministic Kaczmarz Algorithm for Solving Linear Systems

Pseudoinverse-free randomized block iterative algorithms for consistent and inconsistent linear systems

Surrounding the solution of a linear system of equations from all sides

Randomized Kaczmarz iteration methods: Algorithmic extensions and convergence theory

On Multi-Step Randomized Extended Kaczmarz Method for Solving Large Sparse Inconsistent Linear Systems

Paved with good intentions: Analysis of a randomized block Kaczmarz method

On the alternating randomized block Kaczmarz method

Finite Convergence of Circumcentered-Reflection Method on Closed Polyhedral Cones in Euclidean Spaces

A Novel Greedy Kaczmarz Method For Solving Consistent Linear Systems

Randomized Complete Pivoting for Solving Symmetric Indefinite Linear Systems

Randomized Iterative Solver as Iterative Refinement: A Simple Fix Towards Backward Stability

A successive centralized circumcenter reflection method for the convex feasibility problem

On the Convergence of the Randomized Block Kaczmarz Algorithm for Solving a Matrix Equation

Three double multi-step randomized extended Kaczmarz methods for solving large sparse inconsistent linear systems

Randomized Methods for Linear Constraints: Convergence Rates and Conditioning

Randomized Kaczmarz method with adaptive stepsizes for inconsistent linear systems

Randomized block Kaczmarz method with projection for solving least squares