Abstract:In recent years fused-multiply-add (FMA) units with lower-precision multiplications and higher-precision accumulation have proven useful in machine learning/artificial intelligence applications, most notably in training deep neural networks due to their extreme computational intensity. Compared to classical IEEE-754 32 bit (FP32) and 64 bit (FP64) arithmetic, these reduced precision arithmetic can naturally be sped up disproportional to their shortened width. The common strategy of all major hardware vendors is to aggressively further enhance their performance disproportionately. One particular FMA operation that multiplies two BF16 numbers while accumulating in FP32 has been found useful in deep learning, where BF16 is the 16-bit floating point datatype with IEEE FP32 numerical range but 8 significant bits of precision. In this paper, we examine the use this FMA unit to implement higher-precision matrix routines in terms of potential performance gain and implications on accuracy. We demonstrate how a decomposition into multiple smaller datatypes can be used to assemble a high-precision result, leveraging the higher precision accumulation of the FMA unit. We first demonstrate that computations of vector inner products and by natural extension, matrix-matrix products can be achieved by decomposing FP32 numbers in several BF16 numbers followed by appropriate computations that can accommodate the dynamic range and preserve accuracy compared to standard FP32 computations, while projecting up to $5.2 imes$ speed-up. Furthermore, we examine solution of linear equations formulated in the residual form that allows for iterative refinement. We demonstrate that the solution obtained to be comparable to those offered by FP64 under a large range of linear system condition numbers.

Efficient numerical algorithms for multi-precision and multi-accuracy calculation of the error functions and Dawson integral with complex arguments

Remark on "Algorithm 916: Computing the Faddeyeva and Voigt functions": Efficiency Improvements and Fortran Translation

Efficient Evaluation of Double Surface Integrals in Time-Domain Integral Equation Formulations

Effective Quadratic Error Bounds for Floating-Point Algorithms Computing the Hypotenuse Function

Inexactness and Correction of Floating-Point Reciprocal, Division and Square Root

Extension of accurate numerical algorithms for matrix multiplication based on error-free transformation

Computation of polynomial and rational approximations in complex domains by the -method

Detecting and fixing precision-specific operations for measuring floating-point errors.

Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations

High-precision computation: Mathematical physics and dynamics

Leveraging Mixed Precision in Exponential Time Integration Methods

Mixed-precision finite element kernels and assembly: Rounding error analysis and hardware acceleration

Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers

Efficient implementation of elementary functions in the medium-precision range

Adaptive Finite Differencing in High Accuracy Electronic Structure Calculations

Fast Sound Error Bounds for Mixed-Precision Real Evaluation

Floating-Point Numbers with Error Estimates (revised)

Efficient scaling and squaring method for the matrix exponential

Algorithms and data structures for numerical computations with automatic precision estimation

Sound Approximation of Programs with Elementary Functions

Reducing roundoff errors in numerical integration of planetary ephemeris