Abstract:The Shannon entropy of a random variable X has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different random variables contain. This construction is sufficient to derive many measure-theoretical counterparts to information quantities such as the mutual information $I(X; Y) = \mu(\tilde{X} \cap \tilde{Y})$, the joint entropy $H(X,Y) = \mu(\tilde{X} \cup \tilde{Y})$, and the conditional entropy $H(X|Y) = \mu(\tilde{X} \setminus \tilde{Y})$. Here we provide concrete characterisations of these abstract sets and a corresponding signed measure, and in doing so we demonstrate that there exists a much finer decomposition with intuitive properties which we call the logarithmic decomposition (LD). We show that this signed measure space has the useful property that its logarithmic atoms are easily characterised with negative or positive entropy, while also being consistent with Yeung's I-measure. We present the usability of our approach by re-examining the Gács-Körner common information and the Wyner common information from this new geometric perspective and characterising it in terms of our logarithmic atoms - a property we call logarithmic decomposability. We present possible extensions of this construction to continuous probability distributions before discussing implications for quality-led information theory. Lastly, we apply our new decomposition to examine the Dyadic and Triadic systems of James and Crutchfield and show that, in contrast to the I-measure alone, our decomposition is able to qualitatively distinguish between them.

Signed λ-measures and their decompositions

The Logarithmic Minkowski Problem For Non-Symmetric Measures

Quasisymmetric Geometry of the Julia Sets of McMullen Maps

Measures Defined on Quantum Logics of Sets

A Logarithmic Decomposition and a Signed Measure Space for Entropy

SIGNED Q-Analogs OF TORNHEIM'S DOUBLE SERIES

Non-commutative Lebesgue decomposition of non-commutative measures

A Logarithmic Decomposition for Information

A Decomposition of Complex Monge-Ampere Measures

Sign sequences and decomposition numbers

On Decomposable Measures Constructed by Using Stationary Fuzzy Pseudo-Ultrametrics

An abstract decomposition of measures and its many applications

Hausdorff centred measure of non-symmetry cantor sets

Quantum Fisher information and symmetric logarithmic derivative via anti-commutators

New constructions of signed difference sets

A decomposition theorem for balanced measures

The mean value of the generalized Dirichlet L-functions with the weight of the Gauss sums

$L^q$-spectrum of graph-directed self-similar measures that have overlaps and are essentially of finite type

Sub-signature operators and the Dabrowski-Sitarz-Zalecki type theorems for manifolds with boundary

A signed version of Putnam's homology theory: Lefschetz and zeta functions

Three-variable Mahler Measures and Special Values of L-functions of Modular Forms