Abstract:Quantization is a technique for creating efficient Deep Neural Networks (DNNs), which involves performing computations and storing tensors at lower bit-widths than f32 floating point precision. Quantization reduces model size and inference latency, and therefore allows for DNNs to be deployed on platforms with constrained computational resources and real-time systems. However, quantization can lead to numerical instability caused by roundoff error which leads to inaccurate computations and therefore, a decrease in quantized model accuracy. Similarly to prior works, which have shown that both biases and activations are more sensitive to quantization and are best kept in full precision or quantized with higher bit-widths, we show that some weights are more sensitive than others which should be reflected on their quantization bit-width. To that end we propose MixQuant, a search algorithm that finds the optimal custom quantization bit-width for each layer weight based on roundoff error and can be combined with any quantization method as a form of pre-processing optimization. We show that combining MixQuant with BRECQ, a state-of-the-art quantization method, yields better quantized model accuracy than BRECQ alone. Additionally, we combine MixQuant with vanilla asymmetric quantization to show that MixQuant has the potential to optimize the performance of any quantization technique.

Augmented quantization: a general approach to mixture models

Gaussian mixture density modeling and decomposition with weighted likelihood

Quantization for a Mixture of Uniform Distributions Associated with Probability Vectors

Bayesian estimation and prediction for certain mixtures

Quantum Clustering and Gaussian Mixtures

Huber-energy measure quantization

Risk quantization by magnitude and propensity

Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior

Quantization for a set of discrete distributions on the set of natural numbers

Uniform decomposition of probability measures: quantization, clustering and rate of convergence

FunQuant: A R package to perform quantization in the context of rare events and time-consuming simulations

Randomized Quantization: A Generic Augmentation for Data Agnostic Self-supervised Learning

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Conditional quantization for uniform distributions on line segments and regular polygons

Soft Quantization using Entropic Regularization

Maximum entropy and quantized metric models for absolute category ratings

Factor Model of Mixtures

Enhancing Channel Estimation in Quantized Systems with a Generative Prior

Quantizing rare random maps: application to flooding visualization

MixQuant: Mixed Precision Quantization with a Bit-width Optimization Search