Abstract:Since its introduction in 2011, the partial information decomposition (PID) has triggered an explosion of interest in the field of multivariate information theory and the study of emergent, higher-order ("synergistic") interactions in complex systems. Despite its power, however, the PID has a number of limitations that restrict its general applicability: it scales poorly with system size and the standard approach to decomposition hinges on a definition of "redundancy", leaving synergy only vaguely defined as "that information not redundant." Other heuristic measures, such as the O-information, have been introduced, although these measures typically only provided a summary statistic of redundancy/synergy dominance, rather than direct insight into the synergy itself. To address this issue, we present an alternative decomposition that is synergy-first, scales much more gracefully than the PID, and has a straightforward interpretation. Our approach defines synergy as that information in a set that would be lost following the minimally invasive perturbation on any single element. By generalizing this idea to sets of elements, we construct a totally ordered "backbone" of partial synergy atoms that sweeps systems scales. Our approach starts with entropy, but can be generalized to the Kullback-Leibler divergence, and by extension, to the total correlation and the single-target mutual information. Finally, we show that this approach can be used to decompose higher-order interactions beyond just information theory: we demonstrate this by showing how synergistic combinations of pairwise edges in a complex network supports signal communicability and global integration. We conclude by discussing how this perspective on synergistic structure (information-based or otherwise) can deepen our understanding of part-whole relationships in complex systems.

Signed and Unsigned Partial Information Decompositions of Continuous Network Interactions

Partial information decomposition for mixed discrete and continuous random variables

Partial Information Decomposition for Continuous Variables based on Shared Exclusions: Analytical Formulation and Estimation

Generalized decomposition of multivariate information

Exact partial information decompositions for Gaussian systems based on dependency constraints

Invariant components of synergy, redundancy, and unique information among three variables

Unique Information and Secret Key Agreement

The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal

Unique Information via Dependency Constraints

Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework

Capturing and Interpreting Unique Information

Partial Information Decomposition for Data Interpretability and Feature Selection

Orders between channels and implications for partial information decomposition

A scalable, synergy-first backbone decomposition of higher-order structures in complex systems

Computing Unique Information for Poisson and Multinomial Systems

Partial Information Decomposition: Redundancy as Information Bottleneck

Gaussian Partial Information Decomposition: Bias Correction and Application to High-dimensional Data

Characterising directed and undirected metrics of high-order interdependence

The identity of information: how deterministic dependencies constrain information synergy and redundancy