Abstract:SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 1, Page 212-239, March 2023. We propose a new deterministic Kaczmarz algorithm for solving consistent linear systems [math]. Basically, the algorithm replaces orthogonal projections with reflections in the original scheme of Stefan Kaczmarz. Building on this, we give a geometric description of solutions of linear systems. Suppose [math] is [math], we show that the algorithm generates a series of points distributed with patterns on an [math]-sphere centered on a solution. These points lie evenly on [math] lower-dimensional spheres [math], with the property that for any [math], the midpoint of the centers of [math] is exactly a solution of [math]. With this discovery, we prove that taking the average of [math] points on any [math] effectively approximates a solution up to relative error [math], where [math] characterizes the eigengap of the orthogonal matrix produced by the product of [math] reflections generated by the rows of [math]. We also analyze the connection between [math] and [math], the condition number of [math]. In the worst case [math], while for random matrices [math] on average. Finally, we prove that the algorithm indeed solves the linear system [math], where [math] is the lower-triangular matrix such that [math]. The connection between this linear system and the original one is studied. The numerical tests indicate that this new Kaczmarz algorithm has comparable performance to randomized (block) Kaczmarz algorithms.

Randomized Average Kaczmarz Algorithm for Tensor Linear Systems

Randomized Kaczmarz for tensor linear systems

The accelerated tensor Kaczmarz algorithm with adaptive parameters for solving tensor systems

Randomized extended average block Kaczmarz method for inconsistent tensor equations under t-product

Block Matrix and Tensor Randomized Kaczmarz Methods for Linear Feasibility Problems

Randomized sketched TT-GMRES for linear systems with tensor structure

Randomized regularized extended Kaczmarz algorithms for tensor recovery

A Randomized Block Krylov Method for Tensor Train Approximation

Solving the system of nonsingular tensor equations via randomized Kaczmarz-like method

Randomized algorithms for Kroncecker tensor decomposition and applications

Faster randomized block sparse Kaczmarz by averaging

Randomized Kaczmarz with tail averaging

Randomized Kaczmarz algorithm with averaging and block projection

An Accelerated Block Randomized Kaczmarz Method

Regularized Kaczmarz Algorithms for Tensor Recovery

A Deterministic Kaczmarz Algorithm for Solving Linear Systems

Randomized Kaczmarz Algorithm for Inconsistent Linear Systems: An Exact MSE Analysis

A Weighted Randomized Sparse Kaczmarz Method for Solving Linear Systems

On global randomized block Kaczmarz algorithm for solving large-scale matrix equations