Abstract:Recently, low bit-width quantization (e.g., INT8) has been commonly used in deep neural network inference acceleration, but fewer researchers have focused on low-precision training quantization techniques. Considering that the backward propagation in deep neural network training is more computationally intensive and has a heavier energy overhead than the inference process, the quantization of backward propagation is of great interest for the training of very large-scale neural networks as well as for low-power devices with online training requirements. However, the shape specificity and continuous variability of the gradient distribution make gradient quantization difficult, and many studies propose various complex quantization methods for the gradient to reduce training accuracy loss. In this paper, we propose two innovative techniques mainly for INT8 quantization training, including the Data-aware Dynamic Segmentation Quantization scheme to quantize various special gradient distributions and the Update Direction Periodic Search strategy to achieve lower quantization errors. Then, we build a distribution-aware INT8 quantization training framework based on these two methods and conduct experiments on various models and tasks. Experimental results show that our proposed INT8 quantization training method achieves a negligible loss in final training accuracy compared to the full-precision floating-point counterpart on different models, including ResNet, MobileNetV2, VGG, AlexNet, and LSTM. By replacing floating-point computing with 8-bit integer computing for network training, this INT8 quantization training framework provides the possibility of deploying online training directly on low-power devices in the future.

GCINT: Dynamic Quantization Algorithm for Training Graph Convolution Neural Networks Using Only Integers

Optimizing Quantized Neural Networks in a Weak Curvature Manifold

Hessian-based Mixed-Precision Quantization with Transition Aware Training for Neural Networks

Quantized Graph Neural Networks for Image Classification

Training High-Performance and Large-Scale Deep Neural Networks with Full 8-Bit Integers.

Tango: rethinking quantization for graph neural network training on GPUs

Towards Unified INT8 Training for Convolutional Neural Network

Gradient Distribution-aware INT8 Training for Neural Networks

Towards Accurate and Efficient Sub-8-Bit Integer Training

A 4-Bit Integer-Only Neural Network Quantization Method Based on Shift Batch Normalization

QEGCN: an FPGA-based Accelerator for Quantized GCNs with Edge-Level Parallelism

$\rm A^2Q$: Aggregation-Aware Quantization for Graph Neural Networks

Integer-Only Neural Network Quantization Scheme Based on Shift-Batch-Normalization

Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations

HAWQV3: Dyadic Neural Network Quantization

Quantization Networks

Fully Integer-Based Quantization for Mobile Convolutional Neural Network Inference

QGABS: GPU Tensor Core-accelerated Quantized Graph Neural Network based on Adaptive Batch Size

Deep quantization generative networks

Alternating Multi-bit Quantization for Recurrent Neural Networks

Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation