Abstract:The multi-step inertial randomized Kaczmarz (MIRK) method is an iterative method for solving large-scale linear systems. In this paper, we enhance the MIRK method by incorporating the greedy probability criterion, coupled with the introduction of a tighter threshold parameter for this criterion. We prove that the proposed greedy MIRK (GMIRK) method enjoys an improved deterministic linear convergence compared to both the MIRK method and the greedy randomized Kaczmarz method. Furthermore, we exhibit that the multi-step inertial extrapolation approach can be geometrically interpreted as an orthogonal projection method, and establish its relationship with the sketch-and-project method in (SIAM J. Matrix Anal. Appl. 36(4):1660-1690, 2015) and the oblique projection technique in (Results Appl. Math. 16:100342, 2022). Numerical experiments are provided to confirm our results.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to improve the convergence speed and efficiency of large - scale linear system solving methods. Specifically, an improved multi - step inertial randomized Kaczmarz (MIRK) method, namely the greedy multi - step inertial randomized Kaczmarz (GMIRK) method, is proposed in the paper. By introducing the greedy probability criterion and a tighter threshold parameter, the process of solving the linear system is accelerated.
### Background and Motivation
The solution of the linear system \( Ax = b \) has wide applications in scientific computing and engineering fields, such as computed tomography, signal processing, optimal control, and machine learning. The traditional Kaczmarz method is an effective row - operation method, but its convergence speed may be slow, especially when the row order of the coefficient matrix is not good in some cases. To overcome this problem, researchers have proposed a series of improved methods, such as the randomized Kaczmarz (RK) method, the greedy randomized Kaczmarz (GRK) method, and the multi - step inertial randomized Kaczmarz (MIRK) method.
### Main Contributions
1. **Greedy Probability Criterion**: The paper introduces the greedy probability criterion. By excluding the terms with small amplitudes in the residual vector to select the working row, progress is made in each iteration step, and the convergence speed is accelerated.
2. **Tighter Threshold Parameter**: The paper proposes a tighter threshold parameter \(\epsilon_k\), which makes the algorithm theoretically have a faster deterministic linear convergence rate.
3. **Geometric Explanation**: The paper provides a detailed geometric explanation of the multi - step inertial extrapolation method, showing that this method is actually an orthogonal projection on the intersection of two consecutive working rows.
4. **Theoretical Analysis**: The paper proves that the GMIRK method is superior to the MIRK method and the GRK method in terms of the deterministic linear convergence rate.
5. **Numerical Experiments**: Through numerical experiments, the superiority of the GMIRK method in terms of the number of iterations and the calculation time is verified, especially when the rows are highly coherent.
### Method Overview
- **Initialization**: Given the matrix \( A\in\mathbb{R}^{m\times n}\) and the vector \( b\in\mathbb{R}^m\), and the initial point \( x^{(0)}\in\mathbb{R}^n\).
- **Iteration Steps**:
1. Calculate the threshold parameter \(\epsilon_k\).
2. Determine the index set \( I_k\).
3. Select the index \( i_k\) according to a certain probability.
4. Calculate the inertial parameter \(\beta_k\).
5. Update the intermediate point \( w^{(k)}\).
6. Update the iteration point \( x^{(k + 1)}\).
- **Stopping Condition**: Stop the iteration when the relative solution error (RSE) is less than a certain threshold.
### Numerical Experiment Results
The paper conducts numerical experiments with synthetic data and real - world data. The results show that the GMIRK method is significantly superior to the GRK method and the MIRK method in terms of the number of iterations and the calculation time, especially when the rows are highly coherent.
### Conclusion
The GMIRK method proposed in the paper significantly improves the convergence speed and efficiency of large - scale linear system solving by introducing the greedy probability criterion and a tighter threshold parameter. The geometric explanation further reveals the effectiveness of the multi - step inertial extrapolation method, and both the theoretical analysis and the numerical experiment results verify the superiority of this method.