Abstract:This paper presents Mathematical Execution (ME), a new, unified approach for testing numerical code. The key idea is to (1) capture the desired testing objective via a representing function and (2) transform the automated testing problem to the minimization problem of the representing function. The minimization problem is to be solved via mathematical optimization. The main feature of ME is that it directs input space exploration by only executing the representing function, thus avoiding static or symbolic reasoning about the program semantics, which is particularly challenging for numerical code. To illustrate this feature, we develop an ME-based algorithm for coverage-based testing of numerical code. We also show the potential of applying and adapting ME to other related problems, including path reachability testing, boundary value analysis, and satisfiability checking. To demonstrate ME's practical benefits, we have implemented CoverMe, a proof-of-concept realization for branch coverage based testing, and evaluated it on Sun's C math library (used in, for example, Android, Matlab, Java and JavaScript). We have compared CoverMe with random testing and Austin, a publicly available branch coverage based testing tool that supports numerical code (Austin combines symbolic execution and search-based heuristics). Our experimental results show that CoverMe achieves near-optimal and substantially higher coverage ratios than random testing on all tested programs, across all evaluated coverage metrics. Compared with Austin, CoverMe improves branch coverage from 43% to 91%, with significantly less time (6.9 vs. 6058.4 seconds on average).

The NumericalCertification package in Macaulay2

Numerical Simulation for the System of Ordinary Differential Equations

Certifying zeros of polynomial systems using interval arithmetic

The QuillenSuslin Package for Macaulay2

The MatrixSchubert package for Macaulay2

VerifyRealRoots: A Matlab Package for Computing Verified Real Solutions of Polynomials Systems of Equations and Inequalities

Exact Certification in Global Polynomial Optimization Via Sums-of-squares of Rational Functions with Rational Coefficients

Certification of Real Inequalities -- Templates and Sums of Squares

SCMAlgebras: a Macaulay2 package to check sequential Cohen-Macaulayness

A Computer Algebra Package for Polynomial Sequence Recognition

Certifiable Numerical Computations in Schubert Calculus

Coding theory package for Macaulay2

On Sound Compilation of Reals

A Step towards Computational Derived Algebraic Geometry: The RepHomology Package For Macaulay2

Counting solutions of a polynomial system locally and exactly

Effective alpha theory certification using interval arithmetic: alpha theory over regions

Safety Verification of Hybrid Systems Using Certified Multiple Lyapunov-Like Functions.

Certification of minimal approximant bases

Robust Numerical Algebraic Geometry

Mathematical Execution: A Unified Approach for Testing Numerical Code

Numerical Computation of Galois Groups