Abstract:In the field of edge computing, quantizing convolutional neural networks (CNNs) using extremely low bit widths can significantly alleviate the associated storage and computational burdens in embedded hardware, thereby improving computational efficiency. However, such quantization also presents a challenge related to substantial decreases in detection accuracy. This paper proposes an innovative method, called Adaptive Global Power-of-Two Ternary Quantization Based on Unfixed Boundary Thresholds (APTQ). APTQ achieves adaptive quantization by quantizing each filter into two binary subfilters represented as power-of-two values, thereby addressing the accuracy degradation caused by a lack of expression ability of low-bit-width weight values and the contradiction between fixed quantization boundaries and the uneven actual weight distribution. It effectively reduces the accuracy loss while at the same time presenting strong hardware-friendly characteristics because of the power-of-two quantization. This paper extends the APTQ algorithm to propose the APQ quantization algorithm, which can adapt to arbitrary quantization bit widths. Furthermore, this paper designs dedicated edge deployment convolutional computation modules for the obtained quantized models. Through quantization comparison experiments with multiple commonly used CNN models utilized on the CIFAR10, CIFAR100, and Mini-ImageNet data sets, it is verified that the APTQ and APQ algorithms possess better accuracy performance than most state-of-the-art quantization algorithms and can achieve results with very low accuracy loss in certain CNNs (e.g., the accuracy loss of the APTQ ternary ResNet-56 model on CIFAR10 is 0.13%). The dedicated convolutional computation modules enable the corresponding quantized models to occupy fewer on-chip hardware resources in edge chips, thereby effectively improving computational efficiency. This adaptive CNN quantization method, combined with the power-of-two quantization results, strikes a balance between the quantization accuracy performance and deployment efficiency in embedded hardware. As such, valuable insights for the industrial edge computing domain can be gained.

TRQ: Ternary Neural Networks with Residual Quantization

Trained Ternary Quantization

Three-Means Ternary Quantization

Residual Quantization for Low Bit-Width Neural Networks.

HitNet: Hybrid Ternary Recurrent Neural Network

Propagating Asymptotic-Estimated Gradients for Low Bitwidth Quantized Neural Networks

FATNN: Fast and Accurate Ternary Neural Networks

Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation

Designing strong baselines for ternary neural network quantization through support and mass equalization

Pruning Ternary Quantization

Ternary Quantization: A Survey

Sparsification of Deep Neural Networks via Ternary Quantization

QTTNet: Quantized Tensor Train Neural Networks for 3D Object and Video Recognition.

Adaptive Binary-Ternary Quantization

RESIDUAL QUANTIZATION FOR NEURAL NETWORKS

Term Revealing: Furthering Quantization at Run Time on Quantized DNNs

Alternating Multi-bit Quantization for Recurrent Neural Networks

Two-Step Quantization for Low-bit Neural Networks

Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations

Quantization Networks

Adaptive Global Power-of-Two Ternary Quantization Algorithm Based on Unfixed Boundary Thresholds