Abstract:Recurrent Neural Networks (RNNs) have become important tools for tasks such as speech recognition, text generation, or natural language processing. However, their inference may involve up to billions of operations and their large number of parameters leads to large storage size and runtime memory usage. These reasons impede the adoption of these models in real-time, on-the-edge applications. Field-Programmable Gate Arrays (FPGAs) and Application-Specific Integrated Circuits (ASICs) have emerged as promising solutions for the hardware acceleration of these algorithms, thanks to their degree of customization of compute data paths and memory subsystems, which makes them take the maximum advantage from compression techniques for what concerns area, timing, and power consumption. In contrast to the extensive study in compression and quantization for plain feed forward neural networks in the literature, little attention has been paid to reducing the computational resource requirements of RNNs. This work proposes a new effective methodology for the post-training quantization of RNNs. In particular, we focus on the quantization of Long Short-Term Memory (LSTM) RNNs and Gated Recurrent Unit (GRU) RNNs. The proposed quantization strategy is meant to be a detailed guideline toward the design of custom hardware accelerators for LSTM/GRU-based algorithms to be implemented on FPGA or ASIC devices using fixed-point arithmetic only. We applied our methods to LSTM/GRU models pretrained on the IMDb sentiment classification dataset and Penn TreeBank language modelling dataset, thus comparing each quantized model to its floating-point counterpart. The results show the possibility to achieve up to 90% memory footprint reduction in both cases, obtaining less than 1% loss in accuracy and even a slight improvement in the Perplexity per word metric, respectively. The results are presented showing the various trade-offs between memory footprint reduction and accuracy changes, demonstrating the benefits of the proposed methodology even in comparison with other works from the literature.

Resource Reduction of BFGS Quasi-Newton Implementation on FPGA Using Fixed-Point Matrix Updating

Speed and Resource Optimization of BFGS Quasi-Newton Implementation on FPGA Using Inexact Line Search Method for Neural Network Training

A Reconfigurable Approximate Multiplier for Quantized CNN Applications.

FPGA-based Acceleration of Davidon-Fletcher-Powell Quasi-Newton Optimization Method

Fast Neural Network Training on FPGA Using Quasi-Newton Optimization Method

Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs

A Post-training Quantization Method for the Design of Fixed-Point-Based FPGA/ASIC Hardware Accelerators for LSTM/GRU Algorithms

An Efficient FPGA Design for Fixed-point Exponential Calculation.

Optimizing FPGA-Based DNN Accelerator with Shared Exponential Floating-Point Format

Optimization and Implementation of SR T -16 Floating-Point Divider Based on FPGA

A hardware-friendly logarithmic quantization method for CNNs and FPGA implementation

LSFQ: A Low Precision Full Integer Quantization for High-Performance FPGA-Based CNN Acceleration

A neural network accelerated optimization method for FPGA

Floating-point FPGA gaussian elimination in reconfigurable computing system

An FPGA Implementation of Quadratic Programming

A Learning Framework for n-bit Quantized Neural Networks toward FPGAs

AddNet: Deep Neural Networks Using FPGA-Optimized Multipliers

FPGA-Based Hybrid-Type Implementation of Quantized Neural Networks for Remote Sensing Applications

Reconfigurable Custom Floating-Point Instructions (abstract Only)

New Quantization Error Assessment Methodology for Fixed-Point Pipeline FFT Processor Design

Explore Efficient LUT-based Architecture for Quantized Convolutional Neural Networks on FPGA