Abstract:Large language models (LLMs) have significantly advanced the field of naturallanguage processing, while the expensive memory and computation consumptionimpede their practical deployment. Quantization emerges as one of the mosteffective methods for improving the computational efficiency of LLMs. However,existing ultra-low-bit quantization always causes severe accuracy drops. Inthis paper, we empirically relieve the micro and macro characteristics ofultra-low bit quantization and present a novel Dual-Binarization method forLLMs, namely DB-LLM. For the micro-level, we take both the accuracy advantageof 2-bit-width and the efficiency advantage of binarization into account,introducing Flexible Dual Binarization (FDB). By splitting 2-bit quantizedweights into two independent sets of binaries, FDB ensures the accuracy ofrepresentations and introduces flexibility, utilizing the efficient bitwiseoperations of binarization while retaining the inherent high sparsity ofultra-low bit quantization. For the macro-level, we find the distortion thatexists in the prediction of LLM after quantization, which is specified as thedeviations related to the ambiguity of samples. We propose the Deviation-AwareDistillation (DAD) method, enabling the model to focus differently on varioussamples. Comprehensive experiments show that our DB-LLM not only significantlysurpasses the current State-of-The-Art (SoTA) in ultra-low bit quantization(eg, perplexity decreased from 9.64 to 7.23), but also achieves an additional20% reduction in computational consumption compared to the SOTA method underthe same bit-width. Our code will be released soon.

Approaching 1.53-Db Shaping Gain with LDGM Quantization Codes

Near-Ideal M-ary LDGM Quantization with Recovery

Analytical Framework of LDGM-based Iterative Quantization with Decimation.

Optimizing Gain Codebook of LD-CELP.

A quantization schema with negligible degradation for LDPC decoder.

Implementation of Trellis Shaping on LDPC Coded Modulation DSL Systems

Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

Region-Specific Coarse Quantization with Check Node Awareness in 5G-LDPC Decoding

Min-Sum Approximation Decoding Of Ldpc Codes With Adaptive Non-Uniform Quantization

Improve the Performance of LDPC Coded QAM by Selective Bit Mapping in Terrestrial Broadcasting System

Shaping gain for AWGN channel by non-uniform constellation in LDPC-Coded system

Practical Dirty Paper Coding with Nested Binary LDGM-LDPC Codes

LCQ: Low-Rank Codebook based Quantization for Large Language Models

Enhancing Two-Stage Multiple Description Scalar Quantization.

Online variable-length source coding for minimum bitrate LQG control

Adaptive Quantized and Normalized MSA and Its Application for DTMB-A LDPC Codes

Memory-Assisted Quantized LDPC Decoding

DB-LLM: Accurate Dual-Binarization for Efficient LLMs

GPTQT: Quantize Large Language Models Twice to Push the Efficiency

Selective bit mapping for Chinese DTMB LDPC coded QAM modulation scheme