Abstract:This paper presents incremental network quantization (INQ), a novel method, targeting to efficiently convert any pre-trained full-precision convolutional neural network (CNN) model into a low-precision version whose weights are constrained to be either powers of two or zero. Unlike existing methods which are struggled in noticeable accuracy loss, our INQ has the potential to resolve this issue, as benefiting from two innovations. On one hand, we introduce three interdependent operations, namely weight partition, group-wise quantization and re-training. A well-proven measure is employed to divide the weights in each layer of a pre-trained CNN model into two disjoint groups. The weights in the first group are responsible to form a low-precision base, thus they are quantized by a variable-length encoding method. The weights in the other group are responsible to compensate for the accuracy loss from the quantization, thus they are the ones to be re-trained. On the other hand, these three operations are repeated on the latest re-trained group in an iterative manner until all the weights are converted into low-precision ones, acting as an incremental network quantization and accuracy enhancement procedure. Extensive experiments on the ImageNet classification task using almost all known deep CNN architectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the efficacy of the proposed method. Specifically, at 5-bit quantization, our models have improved accuracy than the 32-bit floating-point references. Taking ResNet-18 as an example, we further show that our quantized models with 4-bit, 3-bit and 2-bit ternary weights have improved or very similar accuracy against its 32-bit floating-point baseline. Besides, impressive results with the combination of network pruning and INQ are also reported. The code is available at https://github.com/Zhouaojun/Incremental-Network-Quantization.

QNet: an Adaptive Quantization Table Generator Based on Convolutional Neural Network

Pse: Mixed Quantization Framework of Neural Networks for Efficient Deployment

Quantization Networks

JQF: Optimal JPEG Quantization Table Fusion by Simulated Annealing on Texture Images and Predicting Textures

Neural Image Compression with Quantization Rectifier

Focused Quantization for Sparse CNNs

Deep Neural Network Compression With Single and Multiple Level Quantization

Bag of Tricks with Quantized Convolutional Neural Networks for image classification

Picking Up Quantization Steps for Compressed Image Classification

S-Net: a Scalable Convolutional Neural Network for JPEG Compression Artifact Reduction.

Instance-Aware Dynamic Neural Network Quantization

A Convolutional Neural Network-Based Quantization Method for Block Compressed Sensing of Images

Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

CNQ: Compressor‐Based Non‐uniform Quantization of Deep Neural Networks

Learned Image Compression Guided Adaptive Quantization for Perceptual Quality

Channel-Level Variable Quantization Network for Deep Image Compression

Μl2q: an Ultra-Low Loss Quantization Method for DNN Compression

Cnq: Compressor-Based Non-Uniform Quantization Of Deep Neural Networksinspec Keywordsother Keywordskey Words

A Near Lossless Learned Image Coding Network Quantization Approach for Cross-Platform Inference.

Adaptive Layerwise Quantization for Deep Neural Network Compression

Image Compression Based on Octave Convolution and Semantic Segmentation