Abstract:I introduce a new approach to semantic information based upon the influence of erasure operations (interventions) upon distributions of a system's future trajectories through its phase space. Semantic (meaningful) information is distinguished from syntactic information by the property of having some intrinsic causal power on the future of a given system. As Shannon famously stated, syntactic information is a simple property of probability distributions (the elementary Shannon expression), or correlations between two subsystems and thus does not tell us anything about the meaning of a given message. Kolchinsky & Wolpert (2018) introduced a powerful framework for computing semantic information, which employs interventions upon the state of a system (either initial or dynamic) to erase syntactic information that might influence the viability of a subsystem (such as an organism in an environment). In this work I adapt this framework such that rather than using the viability of a subsystem, we simply observe the changes in future trajectories through a system's phase space as a result of informational interventions (erasures or scrambling). This allows for a more general formalisation of semantic information that does not assume a primary role for the viability of a subsystem (to use examples from Kolchinsky & Wolpert (2018), a rock, a hurricane, or a cell). Many systems of interest have a semantic component, such as a neural network, but may not have such an intrinsic connection to viability as living organisms or dissipative structures. Hence this simple approach to semantic information could be applied to any living, non-living or technological system in order to quantify whether a given quantity of syntactic information within it also has semantic or causal power.

Fundamental Properties of Causal Entropy and Information Gain

Measuring Causality by Taking the Directional Symbolic Mutual Information Approach

Causal Leverage Density: A General Approach to Semantic Information

Causal Geometry

On Geometry of Information Flow for Causal Inference

Causation and Information Flow with Respect to Relative Entropy

Embedding entropy: a nonlinear measure of dynamical causality

Complexity as Causal Information Integration

Quantum Entropic Causal Inference

An Exact Theory of Causal Emergence for Linear Stochastic Iteration Systems

Applications of Common Entropy for Causal Inference

Analysing causal structures with entropy

Quantify the Causes of Causal Emergence: Critical Conditions of Uncertainty and Asymmetry in Causal Structure

A unified theory of information transfer and causal relation

Causation, Information, and Physics

Quantitative causality, causality-guided scientific discovery, and causal machine learning

An information-theoretic framework for conditional causality analysis of brain networks

Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms

Homogenizing Entropy Across Different Environmental Conditions: A Universally Applicable Method for Transforming Continuous Variables

Quantifying Quantum Causal Influences

Direct and Indirect Effects -- An Information Theoretic Perspective