Abstract:In recent years fused-multiply-add (FMA) units with lower-precision multiplications and higher-precision accumulation have proven useful in machine learning/artificial intelligence applications, most notably in training deep neural networks due to their extreme computational intensity. Compared to classical IEEE-754 32 bit (FP32) and 64 bit (FP64) arithmetic, these reduced precision arithmetic can naturally be sped up disproportional to their shortened width. The common strategy of all major hardware vendors is to aggressively further enhance their performance disproportionately. One particular FMA operation that multiplies two BF16 numbers while accumulating in FP32 has been found useful in deep learning, where BF16 is the 16-bit floating point datatype with IEEE FP32 numerical range but 8 significant bits of precision. In this paper, we examine the use this FMA unit to implement higher-precision matrix routines in terms of potential performance gain and implications on accuracy. We demonstrate how a decomposition into multiple smaller datatypes can be used to assemble a high-precision result, leveraging the higher precision accumulation of the FMA unit. We first demonstrate that computations of vector inner products and by natural extension, matrix-matrix products can be achieved by decomposing FP32 numbers in several BF16 numbers followed by appropriate computations that can accommodate the dynamic range and preserve accuracy compared to standard FP32 computations, while projecting up to $5.2 imes$ speed-up. Furthermore, we examine solution of linear equations formulated in the residual form that allows for iterative refinement. We demonstrate that the solution obtained to be comparable to those offered by FP64 under a large range of linear system condition numbers.

MATLAB Simulator of Level-Index Arithmetic

Monotonicity of Multi-Term Floating-Point Adders

Inexactness and Correction of Floating-Point Reciprocal, Division and Square Root

Iterative Methods at Lower Precision

Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations

Application-specific number representation

Floating-point to Fixed-point Transformation Using Extreme Value Theory

Procrastination Is All You Need: Exponent Indexed Accumulators for Floating Point, Posits and Logarithmic Numbers

Improving MATLAB's isprime performance without arbitrary-precision arithmetic

Efficient implementation of elementary functions in the medium-precision range

Arb: Efficient Arbitrary-Precision Midpoint-Radius Interval Arithmetic

INT-FP-QSim: Mixed Precision and Formats For Large Language Models and Vision Transformers

Half precision wave simulation

On the precision attainable with various floating-point number systems

Numerical integration in arbitrary-precision ball arithmetic

Optimizing Logarithmic Arithmetic on FPGAs

The Limits of Mathematics---Tutorial Version

Rigorous Error Analysis for Logarithmic Number Systems

Floating point numbers are real numbers

Elfficient Exact Arithmetic Over Constructive Reals