Abstract:The Shannon entropy of a random variable X has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different random variables contain. This construction is sufficient to derive many measure-theoretical counterparts to information quantities such as the mutual information $I(X; Y) = \mu(\tilde{X} \cap \tilde{Y})$, the joint entropy $H(X,Y) = \mu(\tilde{X} \cup \tilde{Y})$, and the conditional entropy $H(X|Y) = \mu(\tilde{X} \setminus \tilde{Y})$. Here we provide concrete characterisations of these abstract sets and a corresponding signed measure, and in doing so we demonstrate that there exists a much finer decomposition with intuitive properties which we call the logarithmic decomposition (LD). We show that this signed measure space has the useful property that its logarithmic atoms are easily characterised with negative or positive entropy, while also being consistent with Yeung's I-measure. We present the usability of our approach by re-examining the Gács-Körner common information and the Wyner common information from this new geometric perspective and characterising it in terms of our logarithmic atoms - a property we call logarithmic decomposability. We present possible extensions of this construction to continuous probability distributions before discussing implications for quality-led information theory. Lastly, we apply our new decomposition to examine the Dyadic and Triadic systems of James and Crutchfield and show that, in contrast to the I-measure alone, our decomposition is able to qualitatively distinguish between them.

Generalized Deng entropy and its physical meaning

A Modified Belief Entropy in Dempster-Shafer Framework

Transformation of basic probability assignments to probabilities based on a new entropy measure

An Improved Belief Entropy and Its Application in Decision-Making

A Dual Measure of Uncertainty: The Deng Extropy

An Improved Belief Entropy in Evidence Theory

D numbers theory: a generalization of Dempster-Shafer evidence theory

Fractional Deng Entropy and Extropy and Some Applications

Measures of extended fractional Deng entropy and extropy with applications

Generalized information entropy and generalized information dimension

A unified formulation of entropy and its application

The unified extropy and its versions in classical and Dempster–Shafer theories

The maximum entropy negation of basic probability assignment

Plausibility Extropy: The Complementary Dual of Plausibility Entropy

A total uncertainty measure for D numbers based on belief intervals

A Novel Distance Matric: Generalized Relative Entropy

Generalization of Dempster-Shafer theory: A complex belief function

Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective

A Logarithmic Decomposition and a Signed Measure Space for Entropy

Generalised Entropy Accumulation

Performance evaluation of information fusion systems based on belief entropy