An Intrinsic Characterization of Shannon's and Rényi's Entropy

Martin Schlather,Carmen Ditscheid
DOI: https://doi.org/10.3390/e26121051
IF: 2.738
2024-12-05
Entropy
Abstract:All characterizations of the Shannon entropy include the so-called chain rule, a formula on a hierarchically structured probability distribution, which is based on at least two elementary distributions. We show that the chain rule can be split into two natural components, the well-known additivity of the entropy in case of cross-products and a variant of the chain rule that involves only a single elementary distribution. The latter is given as a proportionality relation and, hence, allows a vague interpretation as self-similarity, hence intrinsic property of the Shannon entropy. Analogous characterizations are given for the Rényi entropy and its limits, the min-entropy and the Hartley entropy.
physics, multidisciplinary
What problem does this paper attempt to address?