Abstract:The rapid advancement in AI requires efficient accelerators for training on edge devices, which often face challenges related to the high hardware costs of floating-point arithmetic operations. To tackle these problems, efficient floating-point formats inspired by block floating-point (BFP), such as Microsoft Floating Point (MSFP) and FlexBlock (FB), are emerging. However, they have limited dynamic range and precision for the smaller magnitude values within a block due to the shared exponent. This limits the BFP's ability to train deep neural networks (DNNs) with diverse datasets. This paper introduces the hybrid precision (HPFP) selection algorithms, designed to systematically reduce precision and implement hybrid precision strategies, thereby balancing layer-wise arithmetic operations and data path precision to address the shortcomings of traditional floating-point formats. Reducing the data bit width with HPFP allows more read/write operations from memory per cycle, thereby decreasing off-chip data access and the size of on-chip memories. Unlike traditional reduced precision formats that use BFP for calculating partial sums and accumulating those partial sums in 32-bit Floating Point (FP32), HPFP leads to significant hardware savings by performing all multiply and accumulate operations in reduced floating-point format. For evaluation, two training accelerators for the YOLOv2-Tiny model were developed, employing distinct mixed precision strategies, and their performance was benchmarked against an accelerator utilizing a conventional brain floating point of 16 bits (Bfloat16). The HPFP selection, employing 10 bits for the data path of all layers and for the arithmetic of layers requiring low precision, along with 12 bits for layers requiring higher precision, results in a 49.4% reduction in energy consumption and a 37.5% decrease in memory access. This is achieved with only a marginal mean Average Precision (mAP) degradation of 0.8% when compared to an accelerator based on Bfloat16. This comparison demonstrates that the proposed accelerator based on HPFP can be an efficient approach to designing compact and low-power accelerators without sacrificing accuracy.

Floating-Point Formats and Arithmetic for Highly Accurate Multi-Layer Perceptrons

Nanoscale Design of Multi-Layer Perceptrons Using Floating-Point Arithmetic Units

ASIC Design of Nanoscale Artificial Neural Networks for Inference/Training by Floating-Point Arithmetic

Rethinking Floating Point Overheads for Mixed Precision DNN Accelerators

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Optimizing FPGA-Based DNN Accelerator with Shared Exponential Floating-Point Format

An Arithmetic Unit and Multiplying Accumulation Unit of a Custom Floating Point Data Format

Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations

A Design Framework for Hardware-Efficient Logarithmic Floating-Point Multipliers

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

Optimal Architecture of Floating-Point Arithmetic for Neural Network Training Processors

A Logarithmic Floating-Point Multiplier for the Efficient Training of Neural Networks

Floating-Point Quantization Analysis of Multi-Layer Perceptron Artificial Neural Networks

DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs

Hybrid Precision Floating-Point (HPFP) Selection to Optimize Hardware-Constrained Accelerator for CNN Training

A Configurable Floating-Point Multiple-Precision Processing Element for HPC and AI Converged Computing

Accuracy Booster: Enabling 4-bit Fixed-point Arithmetic for DNN Training

A Stochastic Rounding-Enabled Low-Precision Floating-Point MAC for DNN Training

LBFP: Logarithmic Block Floating Point Arithmetic for Deep Neural Networks

Low Precision Floating Point Arithmetic for High Performance FPGA-based CNN Acceleration

Mixed precision in Graphics Processing Unit