Abstract:To find the least squares solution of a very large and inconsistent system of equations, one can employ the extended Kaczmarz algorithm. This method simultaneously removes the error term, such that a consistent system is asymptotically obtained, and applies Kaczmarz iterations for the current approximation of this system. For random corrections of the right hand side and Kaczmarz updates selected at random, convergence to the least squares solution has been shown. We consider the deterministic control strategies, and show convergence to a least squares solution when row and column updates are chosen according to the almost-cyclic or maximal-residual choice.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to find the least - squares solution in very large and inconsistent systems of linear equations. Specifically, when \(b\) in the linear equation system \(Ax = b\) is not in the range of matrix \(A\) (i.e., \(b\notin R(A)\)), making the system inconsistent, the traditional Kaczmarz algorithm cannot converge to the least - squares solution. To overcome this problem, the author introduced the Extended Kaczmarz algorithm (EK). By simultaneously updating the right - hand - side vector and the solution vector, the error term is gradually removed, thereby progressively obtaining a consistent system, and applying the Kaczmarz iteration to approximate the current solution of this system.
The main contributions of the paper are as follows:
1. **Convergence under deterministic control strategies**: The author proved that when using the Maximal - Residual Control Scheme (MREK) and the Almost - Cyclic Control Scheme (ACEK), the Extended Kaczmarz algorithm can converge to the least - squares solution. These control strategies select the rows and columns with the largest residuals for update respectively, in order to reduce the error more effectively.
2. **Theoretical analysis**: Detailed theoretical analysis is provided, including the proof of the algorithm's convergence and the estimation of the convergence rate. In particular, for the MREK and ACEK algorithms, the author proved that under deterministic control strategies, the algorithms can effectively converge to the least - squares solution by establishing some preparatory lemmas.
3. **Comparison with randomized methods**: The paper also discussed the relationship between these deterministic methods and the Randomized Extended Kaczmarz algorithm (REK). Although REK has been proven to converge to the least - squares solution in expectation, deterministic methods may be more practical in some application scenarios because they do not require random selection of rows and columns.
In conclusion, by introducing and analyzing new deterministic control strategies, this paper fills the gap in the existing literature regarding the convergence of the Extended Kaczmarz algorithm in inconsistent systems, and provides new tools and methods for solving large - scale inconsistent systems of linear equations.