Abstract:We propose a simple doubly stochastic block Gauss--Seidel algorithm for solving linear systems of equations. By varying the row partition parameter and the column partition parameter of the coefficient matrix, we recover the Landweber algorithm, the randomized Kaczmarz algorithm, the randomized Gauss--Seidel algorithm, and the doubly stochastic Gauss--Seidel algorithm. For general (consistent or inconsistent) linear systems, we show the exponential convergence of the {\it norms of the expected iterates} via exact formulas. For consistent linear systems, we prove the exponential convergence of the {\it expected norms of the error and the residual}. Numerical experiments are given to illustrate the efficiency of the proposed algorithm.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to propose a new algorithm to effectively solve the linear equation system \( Ax = b \), where \( A \in \mathbb{R}^{m \times n} \) is the coefficient matrix and \( b \in \mathbb{R}^m \) is the constant vector. Specifically, the author proposes a Doubly Stochastic Block Gauss - Seidel (DSBGS) algorithm and proves its convergence theory.
### Main problems and contributions:
1. **Propose a new algorithm**:
- A Doubly Stochastic Block Gauss - Seidel (DSBGS) algorithm for solving linear equation systems is proposed.
- By changing the row - partitioning parameter and the column - partitioning parameter of the coefficient matrix, several existing algorithms can be recovered, such as the Landweber algorithm, the Random Kaczmarz algorithm, the Random Gauss - Seidel algorithm, and the Doubly Random Gauss - Seidel algorithm.
2. **Convergence proof**:
- For general (consistent or inconsistent) linear systems, the exponential convergence of the expected iteration norm is proved.
- For consistent linear systems, the exponential convergence of the expected error norm and the residual norm is proved.
3. **Numerical experiments**:
- The effectiveness of the proposed algorithm is demonstrated through numerical experiments, including experimental results on synthetic data and real - world data.
- Experiments show that appropriate step sizes and block sizes can significantly improve the convergence speed and performance of the algorithm.
### Mathematical formula representation:
- Linear equation system: \[ Ax = b, \quad A \in \mathbb{R}^{m \times n}, \quad b \in \mathbb{R}^m \]
- Optimization problem: \[ \min_{x \in \mathbb{R}^n} f(x) := \frac{1}{2} \left\| A \right\|_F^2 \left\| b - Ax \right\|_2^2 \]
- Convergence formula: For a consistent linear system, the expected value of the error norm satisfies:
\[ \left\| E[x_k - x_0^\star] \right\|_2 \leq \left( \max_{1 \leq i \leq r} \left| 1 - \alpha \frac{\sigma_i^2(A)}{\left\| A \right\|_F^2} \right| \right)^k \left\| x_0 - x_0^\star \right\|_2 \]
### Summary:
The main contribution of this paper is to propose a new Doubly Stochastic Block Gauss - Seidel algorithm and strictly prove its convergence under different conditions. Numerical experiments further verify the effectiveness and superiority of this algorithm.