Abstract:Quantized neural networks are well known for reducing the latency, power consumption, and model size without significant harm to the performance. This makes them highly appropriate for systems with limited resources and low power capacity. Mixed-precision quantization offers better utilization of customized hardware that supports arithmetic operations at different bitwidths. Quantization methods either aim to minimize the compression loss given a desired reduction or optimize a dependent variable for a specified property of the model (such as FLOPs or model size); both make the performance inefficient when deployed on specific hardware, but more importantly, quantization methods assume that the loss manifold holds a global minimum for a quantized model that copes with the global minimum of the full precision counterpart. Challenging this assumption, we argue that the optimal minimum changes as the precision changes, and thus, it is better to look at quantization as a random process, placing the foundation for a different approach to quantize neural networks, which, during the training procedure, quantizes the model to a different precision, looks at the bit allocation as a Markov Decision Process, and then, finds an optimal bitwidth allocation for measuring specified behaviors on a specific device via direct signals from the particular hardware architecture. By doing so, we avoid the basic assumption that the loss behaves the same way for a quantized model. Automatic Mixed-Precision Quantization for Edge Devices (dubbed AMED) demonstrates its superiority over current state-of-the-art schemes in terms of the trade-off between neural network accuracy and hardware efficiency, backed by a comprehensive evaluation.

Quantized Memory-Augmented Neural Networks

Hessian-based Mixed-Precision Quantization with Transition Aware Training for Neural Networks

Leveraging Automated Mixed-Low-Precision Quantization for tiny edge microcontrollers

An Energy-Efficient Architecture for Accelerating Inference of Memory-Augmented Neural Networks

Alternating Multi-bit Quantization for Recurrent Neural Networks

Low Precision Quantization-aware Training in Spiking Neural Networks with Differentiable Quantization Function

QuantBayes: Weight Optimization for Memristive Neural Networks via Quantization-Aware Bayesian Inference

Assessment of inference accuracy and memory capacity of computation-in-memory enabled neural network due to quantized weights, gradients, input and output signals, and memory non-idealities

Memristive Quantized Neural Networks: A Novel Approach to Accelerate Deep Learning On-Chip.

Quantized Neural Network via Synaptic Segregation Based on Ternary Charge‐Trap Transistors

Low-bit Quantization of Recurrent Neural Network Language Models Using Alternating Direction Methods of Multipliers

A Hybrid Weight Quantization Strategy for Memristive Neural Networks

Adaptive Binary-Ternary Quantization

A Survey on Methods and Theories of Quantized Neural Networks

Quantization Networks

Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search

Adaptive Quantization for Deep Neural Network

A 4-Bit Integer-Only Neural Network Quantization Method Based on Shift Batch Normalization

On the Duration, Addressability, and Capacity of Memory-Augmented Recurrent Neural Networks

U-Net Fixed-Point Quantization for Medical Image Segmentation

AMED: Automatic Mixed-Precision Quantization for Edge Devices