Abstract:The Partial Information Decomposition (PID) [<a class="link-https" data-arxiv-id="1004.2515" href="https://arxiv.org/abs/1004.2515">arXiv:1004.2515</a>] provides a theoretical framework to characterize and quantify the structure of multivariate information sharing. A new method (Idep) has recently been proposed for computing a two-predictor PID over discrete spaces. [<a class="link-https" data-arxiv-id="1709.06653" href="https://arxiv.org/abs/1709.06653">arXiv:1709.06653</a>] A lattice of maximum entropy probability models is constructed based on marginal dependency constraints, and the unique information that a particular predictor has about the target is defined as the minimum increase in joint predictor-target mutual information when that particular predictor-target marginal dependency is constrained. Here, we apply the Idep approach to Gaussian systems, for which the marginally constrained maximum entropy models are Gaussian graphical models. Closed form solutions for the Idep PID are derived for both univariate and multivariate Gaussian systems. Numerical and graphical illustrations are provided, together with practical and theoretical comparisons of the Idep PID with the minimum mutual information PID (Immi). [<a class="link-https" data-arxiv-id="1411.2832" href="https://arxiv.org/abs/1411.2832">arXiv:1411.2832</a>] In particular, it is proved that the Immi method generally produces larger estimates of redundancy and synergy than does the Idep method. In discussion of the practical examples, the PIDs are complemented by the use of deviance tests for the comparison of Gaussian graphical models.

Capturing and Interpreting Unique Information

Unique Information and Secret Key Agreement

Computing Unique Information for Poisson and Multinomial Systems

Signed and Unsigned Partial Information Decompositions of Continuous Network Interactions

Invariant components of synergy, redundancy, and unique information among three variables

Quantum Partial Information Decomposition

Partial Information Decomposition for Continuous Variables based on Shared Exclusions: Analytical Formulation and Estimation

Orders between channels and implications for partial information decomposition

Generalized Decomposition of Multivariate Information

Partial Information Decomposition for Data Interpretability and Feature Selection

Partial information decomposition for mixed discrete and continuous random variables

Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional Data

From Babel to Boole: The Logical Organization of Information Decompositions

Extracting Unique Information Through Markov Relations

Partial Information Decomposition: Redundancy as Information Bottleneck

Exact partial information decompositions for Gaussian systems based on dependency constraints

The identity of information: how deterministic dependencies constrain information synergy and redundancy

A Measure of Synergy Based on Union Information