Abstract:The emergence of low precision floating-point arithmetic in computer hardware has led to a resurgence of interest in the use of mixed precision numerical linear algebra. For linear systems of equations, there has been renewed enthusiasm for mixed precision variants of iterative refinement. We consider the iterative solution of large sparse systems using incomplete factorization preconditioners. The focus is on the robust computation of such preconditioners in half precision arithmetic and employing them to solve symmetric positive definite systems to higher precision accuracy; however, the proposed ideas can be applied more generally. Even for well-conditioned problems, incomplete factorizations can break down when small entries occur on the diagonal during the factorization. When using half precision arithmetic, overflows are an additional possible source of breakdown. We examine how breakdowns can be avoided and we implement our strategies within new half precision Fortran sparse incomplete Cholesky factorization software. Results are reported for a range of problems from practical applications. These demonstrate that, even for highly ill-conditioned problems, half precision preconditioners can potentially replace double precision preconditioners, although unsurprisingly this may be at the cost of additional iterations of a Krylov solver.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the computational failure problem encountered during incomplete factorization pre - processing in low - precision arithmetic, especially when using half - precision (16 - bit) floating - point operations. Specifically, the article focuses on how to robustly calculate the incomplete factorization pre - processing factors of sparse linear systems in half - precision arithmetic and use these factors to solve symmetric positive - definite systems to achieve higher - precision accuracy. Even for problems with very high condition numbers, through appropriate strategies and methods, low - precision pre - processing factors can replace double - precision pre - processing factors, although this may increase the number of iterations of the Krylov subspace solver.
### Main Problems and Solutions
1. **Computational Failure Problem**:
- **Small Diagonal Elements**: During the factorization process, if very small or negative elements appear on the diagonal, it will lead to computational failure.
- **Overflow Problem**: In half - precision arithmetic, due to the small numerical range, overflow is likely to occur, resulting in computational failure.
2. **Solutions**:
- **Avoid Computational Failure**: The article proposes a series of methods to avoid computational failure, including safely detecting potential overflows, globally modifying the matrix to prevent failure, etc.
- **Safe Detection**: By designing a safe detection mechanism, predict and prevent the occurrence of overflow before it actually occurs.
- **Global Modification**: When incomplete factorization fails, ensure the success of factorization by globally adjusting the matrix (for example, adding a shift term).
### Formula Representation
- **Diagonal Element Check**:
\[
l_{kk} \geq \tau > x_{\min}
\]
where $\tau$ is a threshold parameter set according to the precision used, and $x_{\min}$ is the smallest positive number.
- **Column Scaling Operation**:
\[
l_{ik} \leftarrow \frac{l_{ik}}{l_{kk}}
\]
Ensure that $d = l_{kk} \geq \frac{a}{x_{\max}}$, where $a$ is the element with the largest absolute value in the current pivot column.
- **Update Operation**:
\[
l_{ij} \leftarrow l_{ij} - l_{ik} l_{jk}
\]
Use a safe testing algorithm (such as Algorithm 2.2) to ensure that no overflow occurs.
### Summary
The main contribution of this paper lies in proposing methods for robustly calculating incomplete factorization pre - processing factors in low - precision arithmetic and demonstrating the potential of these methods in solving highly ill - conditioned symmetric positive - definite systems. The feasibility and effectiveness of these methods have been verified through a series of experiments.