The sparse Kaczmarz method with surrogate hyperplane for the regularized basis pursuit problem

Ze Wang,Jun-Feng Yin,Ji-Chen Zhao

2024-06-21

Abstract:The Sparse Kaczmarz method is a famous and widely used iterative method for solving the regularized basis pursuit problem. A general scheme of the surrogate hyperplane sparse Kaczmarz method is proposed. In particular, a class of residual-based surrogate hyperplane sparse Kaczmarz method is introduced and the implementations are well discussed. Their convergence theories are proved and the linear convergence rates are studied and compared in details. Numerical experiments verify the efficiency of the proposed methods.

Numerical Analysis

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to accelerate the convergence rate of the sparse Kaczmarz method in solving the regularized basis pursuit problem. Specifically, the author proposes a sparse Kaczmarz method based on alternative hyperplanes, and particularly introduces two implementation methods of the sparse Kaczmarz method based on alternative hyperplanes using residuals. Through theoretical analysis and numerical experiments, the effectiveness and faster convergence rate of these new methods are verified.

### Problem Background
The regularized basis pursuit problem can be expressed as the following optimization problem:
\[
\min_{x \in \mathbb{R}^n} \lambda \|x\|_1+\frac{1}{2}\|x\|_2^2, \quad \text{s.t. } Ax = b,
\]
where \(A\in\mathbb{R}^{m\times n}\), \(b\in\mathbb{R}^m\), and \(\lambda> 0\). This problem has wide applications in fields such as compressed sensing, image processing, and machine learning.

### Research Motivation
The traditional Kaczmarz method and its randomized version have already performed excellently in solving linear equations, but there is still room for improvement in their convergence rate when solving the regularized basis pursuit problem. For this reason, the author proposes a new sparse Kaczmarz method based on alternative hyperplanes, aiming to accelerate convergence and improve the solution efficiency.

### Main Contributions
1. **Propose the alternative hyperplane sparse Kaczmarz method**: By introducing alternative hyperplanes, the iterative process of the traditional sparse Kaczmarz method is improved.
2. **Implementation methods based on residuals**: Two specific alternative hyperplane sparse Kaczmarz methods based on residuals are proposed, which use all residuals and partial residuals for iteration respectively.
3. **Theoretical analysis**: The convergence of the new method is proved, and its linear convergence rate is studied in detail.
4. **Numerical experiments**: Through applications such as random matrices, SuiteSparse matrices, and image reconstruction, the effectiveness and superiority of the new method are verified.

### Conclusion
The results of numerical experiments show that the sparse Kaczmarz method based on alternative hyperplanes is significantly superior to other methods in both the number of iterations and the calculation time when solving the regularized basis pursuit problem, especially when dealing with large - scale problems.