Abstract:ABSTRACTWriting reliable software is difficult. It becomes even more difficult when writing scientific software involving floating-point numbers. Computers provide numbers with limited precision; when confronted with a real whose precision exceeds that limit, they introduce approximation and error. Numerical analysts have developed sophisticated mathematical techniques for performing error and stability analysis of numerical algorithms. However, these are generally not accessible to application programmers or scientists who often do not have in-depth training in numerical analysis and who thus need more automated techniques to analyze their code. In this paper, we develop a novel, practical technique to help application programmers (or even numerical experts) obtain high-level information regarding the numerical stability and accuracy of their code. Our main insight is that by systematically altering (or perturbing) the underlying numerical calculation, we can uncover potential pitfalls in the numerical code. We propose two complementary perturbations to statistically measure numerical stability: value perturbation and expression perturbation. Value perturbation dynamically replaces the least significant bits of each floating-point value, including intermediate values, with random bits to statistically induce numerical error in the code. Expression perturbation statically changes the numerical expressions in the user program to mathematically equivalent (in the reals, likely not in floating-point numbers), but syntactically different forms. We then compare the executions of these "equivalent" forms to help discover and remedy potential instabilities. Value perturbation can overstate error, while expression perturbation is relatively conservative, so we use value perturbation to generate candidates for expression perturbation. We have implemented our technique, and evaluation results on various programs from the literature and the GNU Scientific Library (GSL) show that our technique is effective and offers a practical alternative for understanding numerical stability in scientific software.

Algorithms that satisfy a stopping criterion, probably

Stopping criteria for nonlinear variational problems: iterative approach

Finite Termination of Inexact Proximal Point Algorithms in Hilbert Spaces

Uncertainty quantification for iterative algorithms in linear models with application to early stopping

Numerical tolerance for spectral decompositions of random matrices

On the rates of convergence of simulation based optimization algorithms for optimal stopping problems

Stopping Criteria for the Conjugate Gradient Algorithm in High-Order Finite Element Methods

Iterative Methods at Lower Precision

The Smoothed Duality Gap as a Stopping Criterion

A note on Probably Certifiably Correct algorithms

Stopping Bayesian Optimization with Probabilistic Regret Bounds

Computational Uncertainty Principle in Nonlinear Ordinary Differential Equations

Convergence of a class of algorithms with the level-set subdifferential error bound

A Convergence Framework for Inexact Nonconvex and Nonsmooth Algorithms and Its Applications to Several Iterations

A More Lenient Stopping Rule for Line Search Algorithms

A Class of Numerical Algorithms for Stochastic Differential Equations with Randomly Varying Truncations

Towards Practical Large-scale Randomized Iterative Least Squares Solvers through Uncertainty Quantification

A Study of the Robustness of Iterative Methods for Linear Systems

Perturbing numerical calculations for statistical analysis of floating-point program (in)stability.

Convergence rates in the Prokhorov metric for assessing uncertainty in ill-posed problems

Silent error detection in numerical time-stepping schemes