Abstract:In recent years fused-multiply-add (FMA) units with lower-precision multiplications and higher-precision accumulation have proven useful in machine learning/artificial intelligence applications, most notably in training deep neural networks due to their extreme computational intensity. Compared to classical IEEE-754 32 bit (FP32) and 64 bit (FP64) arithmetic, these reduced precision arithmetic can naturally be sped up disproportional to their shortened width. The common strategy of all major hardware vendors is to aggressively further enhance their performance disproportionately. One particular FMA operation that multiplies two BF16 numbers while accumulating in FP32 has been found useful in deep learning, where BF16 is the 16-bit floating point datatype with IEEE FP32 numerical range but 8 significant bits of precision. In this paper, we examine the use this FMA unit to implement higher-precision matrix routines in terms of potential performance gain and implications on accuracy. We demonstrate how a decomposition into multiple smaller datatypes can be used to assemble a high-precision result, leveraging the higher precision accumulation of the FMA unit. We first demonstrate that computations of vector inner products and by natural extension, matrix-matrix products can be achieved by decomposing FP32 numbers in several BF16 numbers followed by appropriate computations that can accommodate the dynamic range and preserve accuracy compared to standard FP32 computations, while projecting up to $5.2 imes$ speed-up. Furthermore, we examine solution of linear equations formulated in the residual form that allows for iterative refinement. We demonstrate that the solution obtained to be comparable to those offered by FP64 under a large range of linear system condition numbers.

Multiply-add fused float point unit with on-fly denormalized number processing

A Low Latency High Throughput Multiply-accumulator Unit for Float Point and Integer

Floating-Point Unit Processing Denormalized Numbers

A Low-Power In-Memory Multiplication and Accumulation Array with Modified Radix-4 Input and Canonical Signed Digit Weights

Design of A Fully Pipelined Single-Precision Multiply-Add-Fused Unit

Design of A Double-Precision Floating- Point Multiply-Add-Fused Unit with Consideration of Data Dependence

Optimized Design of a Double-Precision Floating-Point Multiply-Add-dused Unit for Data Dependence.

Design of an Extended Floating-Point Multiply-Add-Fused Unit for Exploiting Instruction-Level Parallelism

A Fused Continuous Floating-Point Mac On Fpga

Optimally Approximated and Unbiased Floating-Point Multiplier with Runtime Configurability

A Low-Cost Floating-Point FMA Unit Supporting Package Operations for HPC-AI Applications

LMM: A Fixed-Point Linear Mapping Based Approximate Multiplier for IoT

Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations

Floating-Point Multiply-Add with Approximate Normalization for Low-Cost Matrix Engines

FFT implementation with multi-operand floating point units

Design of low-power and high-speed multiplier

An Effective Implementation of Dual Path Fused Floating-Point Add-Subtract Unit for Reconfigurable Architectures

PAM: A Piecewise-Linearly-Approximated Floating-Point Multiplier with Unbiasedness and Configurability

Design of floating point multiplier using approximate hybrid Radix-4/ Radix-8 booth encoder for image analysis

An Area- and Energy-Efficient Hybrid Architecture for Floating-Point FFT Computations.

Low-Latency Architecture for Implementing Floating-Point Multiplier and Divider Based on Symmetric-Mapping LUT