Abstract:Information transfer between coupled stochastic dynamics, measured by transfer entropy and information flow, is suggested as a physical process underlying the causal relation of systems. While information transfer analysis has booming applications in both science and engineering fields, critical mysteries about its foundations remain unsolved. Fundamental yet difficult questions concern how information transfer and causal relation originate, what they depend on, how they differ from each other, and if they are created by a unified and general quantity. These questions essentially determine the validity of causal relation measurement via information transfer. Here we pursue to lay a complete theoretical basis of information transfer and causal relation. Beyond the well-known relations between these concepts that conditionally hold, we demonstrate that information transfer and causal relation universally originate from specific information synergy and redundancy phenomena characterized by high-order mutual information. More importantly, our theory analytically explains the mechanisms for information transfer and causal relation to originate, vanish, and differ from each other. Moreover, our theory naturally defines the effect sizes of information transfer and causal relation based on high-dimensional coupling events. These results may provide a unified view of information, synergy, and causal relation to bridge Pearl's causal inference theory in computer science and information transfer analysis in physics.

The relationship between transfer entropy and directed information

Information Entropy of Diffusion Processes on Complex Networks

Transfer entropy between communities in complex networks

Eigenvalue-Based Entropy In Directed Complex Networks

On graph entropy measures based on the number of independent sets and matchings

Inferring A Drive-Response Network From Time Series Of Topological Measures In Complex Networks With Transfer Entropy

Measuring Information Transfer by Dispersion Transfer Entropy

On directed information theory and Granger causality graphs

Deriving pairwise transfer entropy from network structure and motifs

Measurement of Information Transfer Based on Phase Increment Transfer Entropy

Transfer Entropy Coefficient: Quantifying Level of Information Flow Between Financial Time Series

Information Transfer in Social Media

Disentangling high order effects in the transfer entropy

Which bits went where? Past and future transfer entropy decomposition with the information bottleneck

A measurement method for the emergent properties of the evolution of network data based on information entropy

A unified theory of information transfer and causal relation

Information theoretic interpretations for H˞ entropy

Complex-valued information entropy measure for networks with directed links (digraphs). Application to citations by community agents with opposite opinions

Causation and Information Flow with Respect to Relative Entropy

Information flow analysis between traffic flow time series based on the transfer entropy

Transfer Zero-Entropy and Its Application for Capturing Cause and Effect Relationship Between Variables