Abstract:In recent years fused-multiply-add (FMA) units with lower-precision multiplications and higher-precision accumulation have proven useful in machine learning/artificial intelligence applications, most notably in training deep neural networks due to their extreme computational intensity. Compared to classical IEEE-754 32 bit (FP32) and 64 bit (FP64) arithmetic, these reduced precision arithmetic can naturally be sped up disproportional to their shortened width. The common strategy of all major hardware vendors is to aggressively further enhance their performance disproportionately. One particular FMA operation that multiplies two BF16 numbers while accumulating in FP32 has been found useful in deep learning, where BF16 is the 16-bit floating point datatype with IEEE FP32 numerical range but 8 significant bits of precision. In this paper, we examine the use this FMA unit to implement higher-precision matrix routines in terms of potential performance gain and implications on accuracy. We demonstrate how a decomposition into multiple smaller datatypes can be used to assemble a high-precision result, leveraging the higher precision accumulation of the FMA unit. We first demonstrate that computations of vector inner products and by natural extension, matrix-matrix products can be achieved by decomposing FP32 numbers in several BF16 numbers followed by appropriate computations that can accommodate the dynamic range and preserve accuracy compared to standard FP32 computations, while projecting up to $5.2 imes$ speed-up. Furthermore, we examine solution of linear equations formulated in the residual form that allows for iterative refinement. We demonstrate that the solution obtained to be comparable to those offered by FP64 under a large range of linear system condition numbers.

Floating-Point Unit Processing Denormalized Numbers

A Low Latency High Throughput Multiply-accumulator Unit for Float Point and Integer

Multiply-add fused float point unit with on-fly denormalized number processing

A Low-Cost Floating-Point FMA Unit Supporting Package Operations for HPC-AI Applications

Design of A Fully Pipelined Single-Precision Multiply-Add-Fused Unit

FFT implementation with multi-operand floating point units

Design of A Fully Pipelined Single-Precision Floating-Point Unit

An Area- and Energy-Efficient Hybrid Architecture for Floating-Point FFT Computations.

Design of A Double-Precision Floating- Point Multiply-Add-Fused Unit with Consideration of Data Dependence

A Floating-point Coprocessor Configured by a FPGA in a Digital Platform Based on Fixed-point DSP for Power Electronics

A pipelined area-efficient and high-speed reconfigurable processor for floating-point FFT/IFFT and DCT/IDCT computations.

Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations

Optimized Design of a Double-Precision Floating-Point Multiply-Add-dused Unit for Data Dependence.

Designing an IEEE-compliant FPU that supports configurable precision for soft processors

Improving Performance of Floating Point Division on GPU and MIC

Design of a Floating-point Coprocessor and Its Applications in Digital Platform for Power Electronics

Design of an Extended Floating-Point Multiply-Add-Fused Unit for Exploiting Instruction-Level Parallelism

A Reconfigurable Floating-Point Fft Architecture

Design of floating point multiplier using approximate hybrid Radix-4/ Radix-8 booth encoder for image analysis

Low-Latency Architecture for Implementing Floating-Point Multiplier and Divider Based on Symmetric-Mapping LUT

FPnew: An Open-Source Multi-Format Floating-Point Unit Architecture for Energy-Proportional Transprecision Computing