Abstract:Exploring the expected quantizing scheme with suitable mixed-precision policy is the key to compress deep neural networks (DNNs) in high efficiency and accuracy. This exploration implies heavy workloads for domain experts, and an automatic compression method is needed. However, the huge search space of the automatic method introduces plenty of computing budgets that make the automatic process challenging to be applied in real scenarios. In this paper, we propose an end-to-end framework named AutoQNN, for automatically quantizing different layers utilizing different schemes and bitwidths without any human labor. AutoQNN can seek desirable quantizing schemes and mixed-precision policies for mainstream DNN models efficiently by involving three techniques: quantizing scheme search (QSS), quantizing precision learning (QPL), and quantized architecture generation (QAG). QSS introduces five quantizing schemes and defines three new schemes as a candidate set for scheme search, and then uses the Differentiable Neural Architecture Search (DNAS) algorithm to seek the layer- or model-desired scheme from the set. QPL is the first method to learn mixed-precision policies by reparameterizing the bitwidths of quantizing schemes, to the best of our knowledge. QPL optimizes both classification loss and precision loss of DNNs efficiently and obtains the relatively optimal mixed-precision model within limited model size and memory footprint. QAG is designed to convert arbitrary architectures into corresponding quantized ones without manual intervention, to facilitate end-to-end neural network quantization. We have implemented AutoQNN and integrated it into Keras. Extensive experiments demonstrate that AutoQNN can consistently outperform state-of-the-art quantization. For 2-bit weight and activation of AlexNet and ResNet18, AutoQNN can achieve the accuracy results of 59.75% and 68.86%, respectively, and obtain accuracy improvements by up to 1.65% and 1.74%, respectively, compared with state-of-the-art methods. Especially, compared with the full-precision AlexNet and ResNet18, the 2-bit models only slightly incur accuracy degradation by 0.26% and 0.76%, respectively, which can fulfill practical application demands.

Symmetric Quantization Scheme and Approximate Batch Normalization for Quantized Neural Networks Training

Optimal Quantization for Batch Normalization in Neural Network Deployments and Beyond

Propagating Asymptotic-Estimated Gradients for Low Bitwidth Quantized Neural Networks

Training High-Performance and Large-Scale Deep Neural Networks with Full 8-Bit Integers.

A 4-Bit Integer-Only Neural Network Quantization Method Based on Shift Batch Normalization

Integer-Only Neural Network Quantization Scheme Based on Shift-Batch-Normalization

Quantization Networks

Class-based Quantization for Neural Networks

Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks

Designing strong baselines for ternary neural network quantization through support and mass equalization

Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks

Residual Quantization for Low Bit-Width Neural Networks.

Adaptive Gradients and Weight Projection Based on Quantized Neural Networks for Efficient Image Classification

A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks

L1 -Norm Batch Normalization for Efficient Training of Deep Neural Networks

AutoQNN: An End-to-End Framework for Automatically Quantizing Neural Networks

Towards Low-Bit Quantization of Deep Neural Networks with Limited Data.

CNQ: Compressor‐Based Non‐uniform Quantization of Deep Neural Networks

Weight Normalization based Quantization for Deep Neural Network Compression

Cnq: Compressor-Based Non-Uniform Quantization Of Deep Neural Networksinspec Keywordsother Keywordskey Words

KCNN: Kernel-wise Quantization to Remarkably Decrease Multiplications in Convolutional Neural Network.