Abstract:ABSTRACTWriting reliable software is difficult. It becomes even more difficult when writing scientific software involving floating-point numbers. Computers provide numbers with limited precision; when confronted with a real whose precision exceeds that limit, they introduce approximation and error. Numerical analysts have developed sophisticated mathematical techniques for performing error and stability analysis of numerical algorithms. However, these are generally not accessible to application programmers or scientists who often do not have in-depth training in numerical analysis and who thus need more automated techniques to analyze their code. In this paper, we develop a novel, practical technique to help application programmers (or even numerical experts) obtain high-level information regarding the numerical stability and accuracy of their code. Our main insight is that by systematically altering (or perturbing) the underlying numerical calculation, we can uncover potential pitfalls in the numerical code. We propose two complementary perturbations to statistically measure numerical stability: value perturbation and expression perturbation. Value perturbation dynamically replaces the least significant bits of each floating-point value, including intermediate values, with random bits to statistically induce numerical error in the code. Expression perturbation statically changes the numerical expressions in the user program to mathematically equivalent (in the reals, likely not in floating-point numbers), but syntactically different forms. We then compare the executions of these "equivalent" forms to help discover and remedy potential instabilities. Value perturbation can overstate error, while expression perturbation is relatively conservative, so we use value perturbation to generate candidates for expression perturbation. We have implemented our technique, and evaluation results on various programs from the literature and the GNU Scientific Library (GSL) show that our technique is effective and offers a practical alternative for understanding numerical stability in scientific software.

A comparison of perturbation analysis techniques

Analytical Assessment for Transient Stability under Stochastic Continuous Disturbances

Computation comparisons of power system dynamics under random excitation

Program of probabilistic stability in power systems and its application

Comparison between CPF and PDIP for voltage stability margin evaluation

Application of Perturbation Analysis to the Design and Analysis of Control Charts

Uncertainty Analysis of Power System Time-Domain Simulation Based on Generalized Polynomial Chaos Method

Application of DTF Method in Disturbance Propagation Analysis of Complex Chemical Process

On Perturbation Propagation for Smoothed Perturbation Analysis

Discontinuous Perturbation Analysis of Discrete-Event Dynamic Systems

Perturbation Analysis in Verification of Discrete-Time Markov Chains.

Perturbation spectrum method for seismic analysis of non-classically damped systems

GSMP formalism and perturbation analysis for discrete event dynamic systems

Quantitative Analysis of Influences of Model Plant Mismatch on Control Loop Behavior

Complexity Analysis for Hybrid Differentiation in Process System Optimization

Perturbing numerical calculations for statistical analysis of floating-point program (in)stability.

Extensions and Generalizations of Smoothed Perturbation Analysis in a Generalized Semi-Markov Process Framework

A Priori Error Estimation for the Stochastic Perturbation Method

Perturbation Amplitudes Design Method Based on Confidence Interval Evaluation for Impedance Measurement

Variance Property of Discontinuous Perturbation Analysis

Program Instability Detection Based on Systematically Optimized Numerical Perturbation