Abstract:There is much interest in the topic of partial information decomposition, both in developing new algorithms and in developing applications. An algorithm, based on standard results from information geometry, was recently proposed by Niu and Quinn (2019). They considered the case of three scalar random variables from an exponential family, including both discrete distributions and a trivariate Gaussian distribution. The purpose of this article is to extend their work to the general case of multivariate Gaussian systems having vector inputs and a vector output. By making use of standard results from information geometry, explicit expressions are derived for the components of the partial information decomposition for this system. These expressions depend on a real-valued parameter which is determined by performing a simple constrained convex optimisation. Furthermore, it is proved that the theoretical properties of non-negativity, self-redundancy, symmetry and monotonicity, which were proposed by Williams and Beer (2010), are valid for the decomposition Iig derived herein. Application of these results to real and simulated data show that the Iig algorithm does produce the results expected when clear expectations are available, although in some scenarios, it can overestimate the level of the synergy and shared information components of the decomposition, and correspondingly underestimate the levels of unique information. Comparisons of the Iig and Idep (Kay and Ince, 2018) methods show that they can both produce very similar results, but interesting differences are provided. The same may be said about comparisons between the Iig and Immi (Barrett, 2015) methods.

Information-Geometric Set Embeddings (IGSE): From Sets to Probability Distributions

Support Vector Machines Based on Information Geometry

Information Geometry of the Probability Simplex: A Short Course

Information Geometry for the Working Information Theorist

A Poisson Decomposition for Information and the Information-Event Diagram

The basics of information geometry

Optimization via Information Geometry

An Interpretable Data Embedding under Uncertain Distance Information.

Geometry Interaction Knowledge Graph Embeddings

FIGCI: Flow-Based Information-Geometric Causal Inference

GAIT: A Geometric Approach to Information Theory

Generalizing Geometric Partition Entropy for the Estimation of Mutual Information in the Presence of Informative Outliers

Geometrical embeddings for distributions into algebras of generalized functions

Information-theoretic inequalities on unimodular Lie groups

Geometric Modeling in Probability and Statistics

Efficient Statistics: Extracting Information from IID Observations

DiriE: Knowledge Graph Embedding with Dirichlet Distribution

Monte Carlo Information Geometry: The dually flat case

A Partial Information Decomposition for Multivariate Gaussian Systems Based on Information Geometry

Set Learning for Generative Information Extraction

Bridging the Space Gap: Unifying Geometry Knowledge Graph Embedding with Optimal Transport