Abstract:In classical information theory, a causal relationship between two variables is typically modelled by assuming that, for every possible state of one of the variables, there exists a particular distribution of states of the second variable. Let us call these two variables the causal and caused variables, respectively. We shall assume that both variables are continuous and one-dimensional. In this work we consider a procedure to transform each variable, using transformations that are differentiable and strictly increasing. We call these increasing transformations. Any causal relationship (as defined here) is associated with a channel capacity, which is the maximum rate that information could be sent if the causal relationship was used as a signalling system. Channel capacity is unaffected when the two variables are changed by use of increasing transformations. For any causal relationship we show that there is always a way to transform the caused variable such that the entropy associated with the caused variable is independent of the value of the causal variable. Furthermore, the resulting universal entropy has an absolute value that is equal to the channel capacity associated with the causal relationship. This observation may be useful in statistical applications. Also, for any causal relationship, it implies that there is a 'natural' way to transform a continuous caused variable. We also show that, with additional constraints on the causal relationship, a natural increasing transformation of both variables leads to a transformed causal relationship that has properties that might be expected from a well-engineered measuring device.

Transfer Zero-Entropy and Its Application for Capturing Cause and Effect Relationship Between Variables

Direct Causality Detection Via the Transfer Entropy Approach

A Causality Capturing Method for Diagnosis Based on Transfer Entropy by Analyzing Trends of Time Series

Detection of Cause-Effect Relations Based on Information Granulation and Transfer Entropy

Detection of Direct Causality Based on Process Data

Measuring Causality by Taking the Directional Symbolic Mutual Information Approach

Data-driven Causal Inference Based on a Modified Transfer Entropy

Detection Of Causality Between Process Variables Based On Industrial Alarm Data Using Transfer Entropy

A Transfer Entropy Method to Quantify Causality in Stochastic Nonlinear Systems**The Authors Would Appreciate the Financial Support from the National Natural Science Foundation of China (61433001) and Tsinghua University Initiative Scientific Research Program.

Homogenizing Entropy Across Different Environmental Conditions: A Universally Applicable Method for Transforming Continuous Variables

A unified theory of information transfer and causal relation

A New Transfer Entropy Approach Based on Information Granulation and Clustering for Root Cause Analysis

Embedding entropy: a nonlinear measure of dynamical causality

Fast and effective pseudo transfer entropy for bivariate data-driven causal inference

Applications of Common Entropy for Causal Inference

Transfer Entropy as a Log-likelihood Ratio

Disentangling high order effects in the transfer entropy

Hidden Variable Discovery Based on Regression and Entropy

On the potential of transfer entropy in turbulent dynamical systems

Some Thoughts on Symbolic Transfer Entropy

Backward transfer entropy: Informational measure for detecting hidden Markov models and its interpretations in thermodynamics, gambling and causality