Abstract:The remarkable success of deep neural networks (DNNs) in various applications is accompanied by a significant increase in network parameters and arithmetic operations. Such increases in memory and computational demands make deep learning prohibitive for resource-constrained hardware platforms such as mobile devices. Recent efforts aim to reduce these overheads, while preserving model performance as much as possible, and include parameter reduction techniques, parameter quantization, and lossless compression techniques. In this chapter, we develop and describe a novel quantization paradigm for DNNs: Our method leverages concepts of explainable AI (XAI) and concepts of information theory: Instead of assigning weight values based on their distances to the quantization clusters, the assignment function additionally considers weight relevances obtained from Layer-wise Relevance Propagation (LRP) and the information content of the clusters (entropy optimization). The ultimate goal is to preserve the most relevant weights in quantization clusters of highest information content. Experimental results show that this novel Entropy-Constrained and XAI-adjusted Quantization (ECQ$^{\text{x}}$) method generates ultra low-precision (2-5 bit) and simultaneously sparse neural networks while maintaining or even improving model performance. Due to reduced parameter precision and high number of zero-elements, the rendered networks are highly compressible in terms of file size, up to $103\times$ compared to the full-precision unquantized DNN model. Our approach was evaluated on different types of models and datasets (including Google Speech Commands, CIFAR-10 and Pascal VOC) and compared with previous work.

A Deep Look into Logarithmic Quantization of Model Parameters in Neural Networks

Optimizing Quantized Neural Networks in a Weak Curvature Manifold

Hessian-based Mixed-Precision Quantization with Transition Aware Training for Neural Networks

Bit Efficient Quantization for Deep Neural Networks

Residual Quantization for Low Bit-Width Neural Networks.

Deep Neural Network Compression With Single and Multiple Level Quantization

Towards the Limit of Network Quantization

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

ECQ$^{\text{x}}$: Explainability-Driven Quantization for Low-Bit and Sparse DNNs

Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats

Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks

Exploring Neural Networks Quantization via Layer-Wise Quantization Analysis

SPFQ: A Stochastic Algorithm and Its Error Analysis for Neural Network Quantization

Adaptive Layerwise Quantization for Deep Neural Network Compression

Learnable Companding Quantization for Accurate Low-bit Neural Networks

Neural Network Language Model Compression with Product Quantization and Soft Binarization

Mixed-Precision Neural Network Quantization Via Learned Layer-Wise Importance

Instance-Aware Dynamic Neural Network Quantization

The Effects Of Quantization On Multi-Layer Feedforward Neural Networks

Quantization Networks

Model compression as constrained optimization, with application to neural nets. Part II: quantization