Abstract:The Shannon entropy of a random variable X has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different random variables contain. This construction is sufficient to derive many measure-theoretical counterparts to information quantities such as the mutual information $I(X; Y) = \mu(\tilde{X} \cap \tilde{Y})$, the joint entropy $H(X,Y) = \mu(\tilde{X} \cup \tilde{Y})$, and the conditional entropy $H(X|Y) = \mu(\tilde{X} \setminus \tilde{Y})$. Here we provide concrete characterisations of these abstract sets and a corresponding signed measure, and in doing so we demonstrate that there exists a much finer decomposition with intuitive properties which we call the logarithmic decomposition (LD). We show that this signed measure space has the useful property that its logarithmic atoms are easily characterised with negative or positive entropy, while also being consistent with Yeung's I-measure. We present the usability of our approach by re-examining the Gács-Körner common information and the Wyner common information from this new geometric perspective and characterising it in terms of our logarithmic atoms - a property we call logarithmic decomposability. We present possible extensions of this construction to continuous probability distributions before discussing implications for quality-led information theory. Lastly, we apply our new decomposition to examine the Dyadic and Triadic systems of James and Crutchfield and show that, in contrast to the I-measure alone, our decomposition is able to qualitatively distinguish between them.

Anqie entropy and arithmetic compactification of natural numbers

On Anqie Entropy and Möbius Disjointness of a Class of Exponential Functions

Entropy of monomial algebras and derived categories

The Categorical Basis of Dynamical Entropy

The entropy function of an invariant measure

Entanglement entropies in the abelian arithmetic Chern-Simons theory

A Logarithmic Decomposition and a Signed Measure Space for Entropy

On Quantifiers for Quantitative Reasoning

Entropy of Non-autonomous Iterated Function Systems

On an entropic analogue of additive energy

A Characterization of Entropy as a Universal Monoidal Natural Transformation

Generalized Rényi entropy accumulation theorem and generalized quantum probability estimation

The Addition Theorem for algebraic entropies induced by non-discrete length functions

Entropy numbers of finite-dimensional Lorentz space embeddings

Signed Rényi Entropy and Quantum Second Laws

Abstract entropy and expansiveness

Quasi-invariant measures concentrating on countable structures

An algebraic generalization of the entropy and its application to statistics

On locally compact groups of small topological entropy

Arithmeticity for Smooth Maximal Rank Positive Entropy Actions of $\mathbb{R}^k$