Abstract:There is much interest in the topic of partial information decomposition, both in developing new algorithms and in developing applications. An algorithm, based on standard results from information geometry, was recently proposed by Niu and Quinn (2019). They considered the case of three scalar random variables from an exponential family, including both discrete distributions and a trivariate Gaussian distribution. The purpose of this article is to extend their work to the general case of multivariate Gaussian systems having vector inputs and a vector output. By making use of standard results from information geometry, explicit expressions are derived for the components of the partial information decomposition for this system. These expressions depend on a real-valued parameter which is determined by performing a simple constrained convex optimisation. Furthermore, it is proved that the theoretical properties of non-negativity, self-redundancy, symmetry and monotonicity, which were proposed by Williams and Beer (2010), are valid for the decomposition Iig derived herein. Application of these results to real and simulated data show that the Iig algorithm does produce the results expected when clear expectations are available, although in some scenarios, it can overestimate the level of the synergy and shared information components of the decomposition, and correspondingly underestimate the levels of unique information. Comparisons of the Iig and Idep (Kay and Ince, 2018) methods show that they can both produce very similar results, but interesting differences are provided. The same may be said about comparisons between the Iig and Immi (Barrett, 2015) methods.

On the Information Dimension of Multivariate Gaussian Processes

On the Increments of A D-Dimensional Gaussian Process

The Self-Intersections of A Gaussian Random Field

LIMIT BEHAVIORS FOR THE INCREMENTS OF A D-Dimensional MULTI-PARAMETER GAUSSIAN PROCESS

Hausdorff Dimension of Set Generated by Exceptional Oscillations of a Class of N-parameter Gaussian Processes

Liminf results on the increments of an (N,d) -Gaussian process

On The Increments Of L(Infinity)-Valued Gaussian Processes

Information theoretic results for stationary time series and the Gaussian-generalized von Mises time series

On the entropy and information of Gaussian mixtures

Multiscale Information Decomposition: Exact Computation for Multivariate Gaussian Processes

High-dimensional Gaussian linear processes: Marchenko-Pastur beyond simultaneous diagonalizability

Geometric Insights into the Multivariate Gaussian Distribution and Its Entropy and Mutual Information

Relative Entropy and Mutual Information in Gaussian Statistical Field Theory

Common Information Dimension

Information loss from dimensionality reduction in 5D-Gaussian spectral data

A Partial Information Decomposition for Multivariate Gaussian Systems Based on Information Geometry

On large increments and liminfs of a two-parameter Gaussian process

Multivariate limit theorems in the context of long-range dependence

A Unifying Framework for Global Gaussianization: Asymptotic Equivalence of Locally Stationary Processes and Bivariate White Noise

The geometry of Gaussian double Markovian distributions

Information Geometry of the Gaussian Space