Abstract:As we reach the limit of Moore's Law, researchers are exploring different paradigms to achieve unprecedented performance. Approximate Computing (AC), which relies on the ability of applications to tolerate some error in the results to trade-off accuracy for performance, has shown significant promise. Despite the success of AC in domains such as Machine Learning, its acceptance in High-Performance Computing (HPC) is limited due to stringent requirements for accuracy. We need tools and techniques to identify regions of code that are amenable to approximations and their impact on the application output quality to guide developers to employ selective approximation. To this end, we propose CHEF-FP, a flexible, scalable, and easy-to-use source-code transformation tool based on Automatic Differentiation (AD) for analyzing approximation errors in HPC applications. CHEF-FP uses Clad, an efficient AD tool built as a plugin to the Clang compiler and based on the LLVM compiler infrastructure, as a backend and utilizes its AD abilities to evaluate approximation errors in C++ code. CHEF-FP works at the source by injecting error estimation code into the generated adjoints. This enables the error-estimation code to undergo compiler optimizations resulting in improved analysis time and reduced memory usage. We also provide theoretical and architectural augmentations to source code transformation-based AD tools to perform FP error analysis. This paper primarily focuses on analyzing errors introduced by mixed-precision AC techniques. We also show the applicability of our tool in estimating other kinds of errors by evaluating our tool on codes that use approximate functions. Moreover, we demonstrate the speedups CHEF-FP achieved during analysis time compared to the existing state-of-the-art tool due to its ability to generate and insert approximation error estimate code directly into the derivative source.

Reverse-Mode AD of Reduce-by-Index and Scan in Futhark

Parallel Dual-Numbers Reverse AD

Forward-Mode Automatic Differentiation of Compiled Programs

Decomposing reverse-mode automatic differentiation

Futhark: purely functional GPU-programming with nested parallelism and in-place array updates

Using Rewrite Strategies for Efficient Functional Automatic Differentiation

Differentiating a Tensor Language

DiffSharp: An AD Library for .NET Languages

A backward automatic differentiation framework for reservoir simulation

AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia

Automatic Differentiation in Prolog

Efficient Forward-Mode Algorithmic Derivatives of Geant4

Assign optimization for algorithmic differentiation reuse index management strategies

A hybrid one‐step leapfrog ADI‐FDTD and subgridding approach based on a heterogeneous computing platform

Fast And Automatic Floating Point Error Analysis With CHEF-FP

An AD based library for Efficient Hessian and Hessian-Vector Product Computation on GPU

The Stan Math Library: Reverse-Mode Automatic Differentiation in C++

FastAD: Expression Template-Based C++ Library for Fast and Memory-Efficient Automatic Differentiation

Step-based checkpointing with high-level algorithmic differentiation

Understanding Automatic Differentiation Pitfalls

On improving the efficiency of ADER methods