Finite Blocklength Lossy Source Coding for Discrete Memoryless Sources
Lin Zhou,Mehul Motani
DOI: https://doi.org/10.1561/0100000134
2023-01-01
Foundations and Trends® in Communications and Information Theory
Abstract:Shannon propounded a theoretical framework (collectively called information theory) that uses mathematical tools to understand, model and analyze modern mobile wireless communication systems. A key component of such a system is source coding, which compresses the data to be transmitted by eliminating redundancy and allows reliable recovery of the information from the compressed version. In modern 5G networks and beyond, finite blocklength lossy source coding is essential to provide ultra-reliable and low-latency communications. The analysis of point-to-point and multiterminal settings from the perspective of finite blocklength lossy source coding is therefore of great interest to 5G system designers and is also related to other long-standing problems in information theory. In this monograph, we review recent advances in second-order asymptotics for lossy source coding, which provides approximations to the finite blocklength performance of optimal codes. The monograph is divided into three parts. In Part I, we motivate the monograph, present basic definitions, introduce mathematical tools and illustrate the motivation of non-asymptotic and second-order asymptotics via the example of lossless source coding. In Part II, we first present existing results for the rate-distortion problem with proof sketches. Subsequently, we present five generations of the rate-distortion problem to tackle various aspects of practical quantization tasks: noisy source, noisy channel, mismatched code, Gauss-Markov source and fixed-to-variable length compression. By presenting theoretical bounds for these settings, we illustrate the effect of noisy observation of the source, the influence of noisy transmission of the compressed information, the effect of using a fixed coding scheme for an arbitrary source and the roles of source memory and variable rate. In Part III, we present four multiterminal generalizations of the rate-distortion problem to consider multiple encoders, decoders or source sequences: the Kaspi problem, the successive refinement problem, the Fu-Yeung problem and the Gray-Wyner problem. By presenting theoretical bounds for these multiterminal problems, we illustrate the role of side information, the optimality of stop and transmit, the effect of simultaneous lossless and lossy compression, and the tradeoff between encoders’ rates in compressing correlated sources. Finally, we conclude the monograph, mention related results and discuss future directions.