Abstract:Quantization has been proven to be a vital method for improving the inference efficiency of deep neural networks (DNNs). However, it is still challenging to strike a good balance between accuracy and efficiency while quantizing DNN weights or activation values from high-precision formats to their quantized counterparts. We propose a new method called elastic significant bit quantization (ESB) that controls the number of significant bits of quantized values to obtain better inference accuracy with fewer resources. We design a unified mathematical formula to constrain the quantized values of the ESB with a flexible number of significant bits. We also introduce a distribution difference aligner (DDA) to quantitatively align the distributions between the full-precision weight or activation values and quantized values. Consequently, ESB is suitable for various bell-shaped distributions of weights and activation of DNNs, thus maintaining a high inference accuracy. Benefitting from fewer significant bits of quantized values, ESB can reduce the multiplication complexity. We implement ESB as an accelerator and quantitatively evaluate its efficiency on FPGAs. Extensive experimental results illustrate that ESB quantization consistently outperforms state-of-the-art methods and achieves average accuracy improvements of 4.78%, 1.92%, and 3.56% over AlexNet, ResNet18, and MobileNetV2, respectively. Furthermore, ESB as an accelerator can achieve 10.95 GOPS peak performance of 1k LUTs without DSPs on the Xilinx ZCU102 FPGA platform. Compared with CPU, GPU, and state-of-the-art accelerators on FPGAs, the ESB accelerator can improve the energy efficiency by up to 65x, 11x, and 26x, respectively.

ADT-FSE: A New Encoder for SZ

Optimizing Huffman Decoding for Error-Bounded Lossy Compression on GPUs

cuSZ-$i$: High-Ratio Scientific Lossy Compression on GPUs with Optimized Multi-Level Interpolation

Accelerating Relative-error Bounded Lossy Compression for HPC Datasets with Precomputation-Based Mechanisms.

High-Ratio Lossy Compression: Exploring the Autoencoder to Compress Scientific Data

Performance Optimization for Relative-Error-Bounded Lossy Compression on Scientific Data.

FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs

Accelerating Lossy Compression on HPC Datasets Via Partitioning Computation for Parallel Processing

SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors

A Versatile Compression Method for Floating-Point Data Stream

An Error-Bounded Lossy Compression Method with Bit-Adaptive Quantization for Particle Data

SDC Resilient Error-bounded Lossy Compressor

High-performance Effective Scientific Error-bounded Lossy Compression with Auto-tuned Multi-component Interpolation

QZFS: QAT Accelerated Compression in File System for Application Agnostic and Cost Efficient Data Storage.

SQuant: On-the-Fly Data-Free Quantization Via Diagonal Hessian Approximation

Machete: an Efficient Lossy Floating-Point Compressor Designed for Time Series Databases

Adaptive Encoding Strategies for Erasing-Based Lossless Floating-Point Compression

FPGA Acceleration of Zstd Compression Algorithm

FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications

ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization.

Elastic Significant Bit Quantization and Acceleration for Deep Neural Networks