Redundancy and optimality of codes for infinite-entropy sources

M. Klimesh
DOI: https://doi.org/10.1109/ISIT.2008.4595329
2008-07-06
Abstract:Redundancy and optimality are standard concepts for prefix codes for finite-entropy sources, but they can be extended to codes for (discrete) infinite-entropy sources as well. Here three redundancy definitions and two optimality definitions are considered that accomplish this. The redundancy definitions turn out to be equivalent, and they are also equivalent to the 'expected codeword length minus entropy' definition when the source entropy is finite. An analogous statement holds for the optimality definitions. A theorem on the redundancy of finite sources is also presented; it provides a simple bound on the difference between the redundancies of optimal codes for two sources that contain an identical subset of symbol probabilities.
What problem does this paper attempt to address?