Abstract:We consider the problem of optimizing the parameter of a two-stage algorithm for approximate solution of a system of linear algebraic equations with a sparse $n\times n$-matrix, i.e., with one in which the number of nonzero elements is $m\!=\!O(n)$. The two-stage algorithm uses conjugate gradient method at its stages. At the 1st stage, an approximate solution with accuracy $\varepsilon_1$ is found for zero initial vector. All numerical values used at this stage are represented as single-precision numbers. The obtained solution is used as initial approximation for an approximate solution with a given accuracy $\varepsilon_2$ that we obtain at the 2nd stage, where double-precision numbers are used. Based on the values of some matrix parameters, computed in a time not exceeding $O(m)$, we need to determine the value $\varepsilon_1$ which minimizes the total computation time at two stages. Using single-precision numbers for computations at the 1st stage is advantageous, since the execution time of one iteration will be approximately half that of one iteration at the 2nd stage. At the same time, using machine numbers with half the mantissa length accelerates the growth of the rounding error per iteration of the conjugate gradient method at the 1st stage, which entails an increase in the number of iterations performed at 2nd stage. As parameters that allow us to determine $\varepsilon_1$ for the input matrix, we use $n$, $m$, an estimate of the diameter of the graph associated with the matrix, an estimate of the spread of the matrix' eigenvalues, and estimates of its maximum eigenvalue. The optimal or close to the optimal value of $\varepsilon_1$ can be determined for matrix with such a vector of parameters using the nearest neighbor regression or some other type of regression.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to optimize the parameter $\varepsilon_1$ in a two - stage algorithm for solving sparse linear algebraic equations (SLAE) in order to minimize the total computation time. Specifically, the problem can be described as follows: ### Problem Description Given a sparse matrix $A$ and a vector $b$, the linear equation system $Ax = b$ needs to be solved. To improve computational efficiency, a mixed - precision iterative method is used. The specific steps are: 1. **First stage**: Use single - precision floating - point numbers to perform the conjugate gradient method (CG), starting from a zero initial vector, and reach the precision $\varepsilon_1$. 2. **Second stage**: Use the approximate solution obtained in the first stage as the initial value, and use double - precision floating - point numbers to perform CG, and finally reach the precision $\varepsilon_2$. ### Objective The objective is to find a suitable value of $\varepsilon_1$ so that the total computation time of the two - stage algorithm is minimized. That is, determine the optimal or near - optimal value of $\varepsilon_1$ to ensure that the computation time is reduced as much as possible while maintaining a certain precision. ### Key Challenges - Using single - precision floating - point numbers can accelerate the calculation in the first stage, but it will increase the rounding error, resulting in more iteration times in the second stage. - It is necessary to predict the optimal value of $\varepsilon_1$ according to some parameters of the matrix (such as the matrix dimension $n$, the number of non - zero elements $m$, the graph diameter, the eigenvalue distribution, etc.). - The estimation of these parameters needs to be completed in $O(m)$ time to ensure that the computational complexity will not be significantly increased. ### Solution The author proposes a regression - based method to predict the optimal value of $\varepsilon_1$. The specific steps include: 1. **Select matrix parameters**: Consider the matrix dimension $n$, the number of non - zero elements $m$, the estimated value of the graph diameter $\tilde{\ell}$, the estimated value of the eigenvalue distribution $\tilde{spr}$, and the estimated value of the maximum eigenvalue $\tilde{\lambda}_{\max}$. 2. **Construct sample set**: Generate different types of sparse matrices to construct training sample sets and test sample sets. 3. **Apply kNN regression**: Use the kNN method to classify the test samples, calculate the total number of iteration times under different values of $\varepsilon_1$, and evaluate the prediction accuracy. ### Experimental Results The experimental results show that the value of $\varepsilon_1$ predicted by the kNN regression method can significantly reduce the total number of iteration times, thereby reducing the computation time. For example, for a test sample containing 1,000 matrices, the total number of iteration times using the $\varepsilon_1$ value predicted by kNN is about 14% less than that of using only double - precision iteration. In summary, this paper aims to achieve efficient computation for solving sparse linear algebraic equations by optimizing the parameter $\varepsilon_1$ in the two - stage algorithm.

Parameter optimization for restarted mixed precision iterative sparse solver

Mixed Precision Iterative Refinement with Adaptive Precision Sparse Approximate Inverse Preconditioning

Solving Dense Linear Systems Faster Than via Preconditioning

Mixed precision sketching for least-squares problems and its application in GMRES-based iterative refinement

Fine-grained Analysis and Faster Algorithms for Iteratively Solving Linear Systems

Faster Linear Systems and Matrix Norm Approximation via Multi-level Sketched Preconditioning

Improving the Performance of the GMRES Method using Mixed-Precision Techniques

Optimal Solutions of Well-Posed Linear Systems via Low-Precision Right-Preconditioned GMRES with Forward and Backward Stabilization

Low-Rank Matrix Iteration Using Polynomial-Filtered Subspace Extraction.

Minimization of the Pseudospectral Abscissa of a Quadratic Matrix Polynomial

Improving Linear System Solvers for Hyperparameter Optimisation in Iterative Gaussian Processes

Single-pass Nyström approximation in mixed precision

On adapting Nesterov's scheme to accelerate iterative methods for linear problems

Matrix Pencil Optimal Iterative Algorithms and Restarted Versions for Linear Matrix Equation and Pseudoinverse

Sparse polynomial approximations for affine parametric saddle point problems

A note on solving nonlinear optimization problems in variable precision

How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization

Combining Sparse Approximate Factorizations with Mixed-precision Iterative Refinement

Iterative Methods in GPU-Resident Linear Solvers for Nonlinear Constrained Optimization

Mixed precision iterative refinement for least squares with linear equality constraints and generalized least squares problems

Novel Optimization Techniques for Parameter Estimation