Abstract:Classical probabilistic rounding error analysis is particularly well suited to stochastic rounding (SR), and it yields strong results when dealing with floating-point algorithms that rely heavily on summation. For many numerical linear algebra algorithms, one can prove probabilistic error bounds that grow as O(nu), where n is the problem size and u is the unit roundoff. These probabilistic bounds are asymptotically tighter than the worst-case ones, which grow as O(nu). For certain classes of algorithms, SR has been shown to be unbiased. However, all these results were derived under the assumption that SR is implemented exactly, which typically requires a number of random bits that is too large to be suitable for practical implementations. We investigate the effect of the number of random bits on the probabilistic rounding error analysis of SR. To this end, we introduce a new rounding mode, limited-precision SR. By taking into account the number r of random bits used, this new rounding mode matches hardware implementations accurately, unlike the ideal SR operator generally used in the literature. We show that this new rounding mode is biased and that the bias is a function of r. As r approaches infinity, however, the bias disappears, and limited-precision SR converges to the ideal, unbiased SR operator. We develop a novel model for probabilistic error analysis of algorithms employing SR. Several numerical examples corroborate our theoretical findings.

Probabilistic Rounding Error Analysis of Modified Gram–Schmidt

Probabilistic error analysis of CholeskyQR based on columns

Statistical Rounding Error Analysis for the Computation of Random Vectors and Matrices

Probabilistic error analysis of limited-precision stochastic rounding

Error Analysis of Sum-Product Algorithms under Stochastic Rounding

Randomized Approximation of the Gram Matrix: Exact Computation and Probabilistic Bounds

Probabilistic Examination of Least Squares Error in Low-bitwidth Cholesky Decomposition

Stochastic Rounding Implicitly Regularizes Tall-and-Thin Matrices

Probabilistic Analysis of Least Squares, Orthogonal Projection, and QR Factorization Algorithms Subject to Gaussian Noise

Rigorous Roundoff Error Analysis of Probabilistic Floating-Point Computations

What You Will Gain By Rounding: Theory and Algorithms for Rounding Rank

Bounds on non-linear errors for variance computation with stochastic rounding

Rounding Errors of Partial Derivatives of Simple Eigenvalues of the Quadratic Eigenvalue Problem.

A stochastic perturbation analysis of the QR decomposition and its applications

A Probabilistic Approach to Floating-Point Arithmetic

Probabilistic perturbation bounds of matrix decompositions

A QLP Decomposition via Randomization.

Average-case analysis of the Gaussian elimination with partial pivoting

Sharper Bounds for Proximal Gradient Algorithms with Errors

A Multi-Precision Quadratic Regularization Method for Unconstrained Optimization with Rounding Error Analysis

On the influence of stochastic roundoff errors and their bias on the convergence of the gradient descent method with low-precision floating-point computation