Abstract:Hardware trends have motivated the development of mixed precision algo-rithms in numerical linear algebra, which aim to decrease runtime while maintaining acceptable accuracy. One recent development is the development of an adaptive precision sparse matrix-vector produce routine, which may be used to accelerate the solution of sparse linear systems by iterative methods. This approach is also applicable to the application of inexact preconditioners, such as sparse approximate inverse preconditioners used in Krylov subspace methods. In this work, we develop an adaptive precision sparse approximate inverse preconditioner and demonstrate its use within a five-precision GMRES-based iterative refinement method. We call this algorithm variant BSPAI-GMRES-IR. We then analyze the conditions for the convergence of BSPAI-GMRES-IR, and determine settings under which BSPAI-GMRES-IR will produce similar backward and forward errors as the existing SPAI-GMRES-IR method, the latter of which does not use adaptive precision in preconditioning. Our numerical experiments show that this approach can potentially lead to a reduction in the cost of storing and applying sparse approximate inverse preconditioners, although a significant reduction in cost may comes at the expense of increasing the number of GMRES iterations required for convergence.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use the Block - Sparse Approximate Inverse preconditioner with Adaptive Precision (BSPAI) combined with the Generalized Minimum Residual Method (GMRES) for Iterative Refinement (IR) when solving large - scale sparse linear systems \(Ax = b\), in order to reduce the cost of storing and applying the sparse approximate inverse preconditioner, while maintaining backward and forward errors comparable to existing methods (such as SPAI - GMRES - IR). ### Background and Motivation With the development of modern hardware, the application of low - precision arithmetic (such as half - precision) in numerical linear algebra has received increasing attention. Low - precision arithmetic can significantly reduce the costs of computation, storage, and data transfer, but it also increases the rounding error and the limitation of the representation range. Therefore, mixed - precision algorithms have emerged. By choosing different precisions at different computational stages, performance, memory, and energy - consumption efficiency can be improved without sacrificing accuracy. ### Research Content The paper proposes a new algorithm variant - BSPAI - GMRES - IR. This algorithm uses the Block - Sparse Approximate Inverse preconditioner with Adaptive Precision and is applied in the GMRES iterative refinement framework with five precisions. The specific steps are as follows: 1. **Preconditioner Construction**: First, use the SPAI algorithm to calculate the sparse approximate inverse preconditioner \(M\) at precision \(u_f\), and transpose it to \(M^T\). 2. **Initial Solution Calculation**: Calculate the initial solution \(x_0 = M^T b\) at precision \(u_f\), and store \(x_0\) at precision \(u\). 3. **Iterative Refinement**: - Calculate the residual \(r_i = b - A x_i\) and store it at precision \(u_r\). - Use the left - preconditioned GMRES to solve the correction equation \(M A d_i = M r_i\), where the matrix - vector multiplication uses the Adaptive - Precision Sparse Matrix - Vector Multiplication (Algorithm 2.3), and calculate the matrix - vector multiplication of \(A\) at precision \(u_p\). - Update the solution \(x_{i + 1}=x_i + d_i\) and store it at precision \(u\). ### Convergence Analysis The paper analyzes the convergence conditions of BSPAI - GMRES - IR and determines under which settings this method can produce backward and forward errors similar to existing methods (such as SPAI - GMRES - IR). Specifically, when the following conditions are met, BSPAI - GMRES - IR can ensure convergence: - **Backward Error**: \(\epsilon_{\text{nw}}\leq(q - 1)u_1 + c\epsilon\) where \(c=(1+(q - 1)u_1)+\max_i\sum_{k = 1}^q p_{ik}^2(1 + u_k)^2\), and \(p_{ik}\) is the number of elements in bucket \(B_{ik}\). - **Forward Error**: \(\frac{\|d_i-\hat{d}_i\|_\infty}{\|d_i\|_\infty}\lesssim f(n,k)\left(u_g+\frac{q u_p+(q - 1)u_1 + c\epsilon}{\kappa_\infty(M)}\right)\kappa_\infty(\tilde{A})\) ### Numerical Experiments The paper evaluates the performance of BSPAI - GMRES - IR through numerical experiments and compares it with SPAI - GMRES - IR. The experimental results show that BSPAI - GMRES - IR can significantly reduce the cost of storing and applying the sparse approximate inverse preconditioner in some cases, although this may increase the number of GMRES iterations. ### Conclusion When solving large - scale sparse linear systems, the BSPAI - GMRES - IR method proposed in the paper can significantly reduce the cost while maintaining high precision through the application of the Block - Sparse Approximate Inverse preconditioner with Adaptive Precision.

Mixed Precision Iterative Refinement with Adaptive Precision Sparse Approximate Inverse Preconditioning

Mixed precision sketching for least-squares problems and its application in GMRES-based iterative refinement

A Study of Mixed Precision Strategies for GMRES on GPUs

A power sparse approximate inverse preconditioning procedure for large sparse linear systems

Projection Improved SPAI Preconditioner for FGMRES

A Residual Based Sparse Approximate Inverse Preconditioning Procedure for Large Sparse Linear Systems

Optimal Solutions of Well-Posed Linear Systems via Low-Precision Right-Preconditioned GMRES with Forward and Backward Stabilization

Combining Sparse Approximate Factorizations with Mixed-precision Iterative Refinement

Mixed Precision FGMRES-Based Iterative Refinement for Weighted Least Squares

Mixed precision iterative refinement for linear inverse problems

N A ] 2 7 A pr 2 01 5 A Residual Based Sparse Approximate Inverse Preconditioning Procedure for Large Sparse Linear Systems ∗

Five-Precision GMRES-Based Iterative Refinement

Generating Approximate Inverse Preconditioners for Sparse Matrices Using CUDA and GPGPU

Improving the Performance of the GMRES Method using Mixed-Precision Techniques

Mixed precision iterative refinement for least squares with linear equality constraints and generalized least squares problems

An Approach to Making SPAI and PSAI Preconditioning Effective for Large Irregular Sparse Linear Systems.

Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers

A transformation approach that makes SPAI, PSAI and RSAI procedures efficient for large double irregular nonsymmetric sparse linear systems

Avoiding breakdown in incomplete factorizations in low precision arithmetic

Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra Software