Abstract:A general theory for computing information transfers in nonlinear stochastic systems driven by deterministic forcing and additive and/or multiplicative noises, is presented. It extends the Liang-Kleeman framework of causality inference to nonlinear cases based on information transfer across system variables (Liang, 2016). We present an effective method of computing formulas of the rates of Shannon entropy transfer (RETs) between selected causal and consequential variables, relying on the estimation from data of conditional expectations of the system forcing and their derivatives. Those expectations are approximated by nonlinear differentiable regressions, leading to a much easier and more robust way of computing RETs than the brute-force approach calling for numerical integrals over the state-space and the knowledge of the multivariate probability density of the system. The approach is fully adapted to the case where no model equations are available, starting with a nonlinear model fitting from data of the consequential variables, and the subsequent application of method to the fitted model. RETs are decomposed into sums of single one-to-one RETs plus synergetic terms (of pure nonlinear nature) accounting for the joint causal effect of groups of variables. State-dependent RET formulas are also introduced, showing where in state-space the entropy transfers and local synergies are more relevant. A comparison of the RETs estimations is performed with previous methods in the context of two models: (i) a model derived from a potential function and (ii) the classical chaotic Lorenz system, both forced by additive and/or multiplicative noises. The analysis demonstrates that the new estimations are robust, cheaper, and less data-demanding, providing evidence of the possibilities and generalizations offered by the method and opening new perspectives on real-world applications.

A Transfer Entropy Method to Quantify Causality in Stochastic Nonlinear Systems**The Authors Would Appreciate the Financial Support from the National Natural Science Foundation of China (61433001) and Tsinghua University Initiative Scientific Research Program.

Identifying Explicit Expression of Response Probability Density of Nonlinear Stochastic System: Information-theoretic Method

Distilling Slow Process Probability Density from Fast Random Data

An Efficient Latent Variable Optimization Approach with Stochastic Constraints for Complex Industrial Process

Data-driven Causal Inference Based on a Modified Transfer Entropy

A general theory to estimate Information transfer in nonlinear systems

Direct Causality Detection Via the Transfer Entropy Approach

A Causality Capturing Method for Diagnosis Based on Transfer Entropy by Analyzing Trends of Time Series

Transfer Zero-Entropy and Its Application for Capturing Cause and Effect Relationship Between Variables

Novel Causal Network Modeling Method Integrating Process Knowledge with Modified Transfer Entropy: A Case Study of Complex Chemical Processes

Nonparametric inference of stochastic differential equations based on the relative entropy rate

Transfer Entropy Between Multivariate Time Series

Detection of Cause-Effect Relations Based on Information Granulation and Transfer Entropy

Information Flow Within Stochastic Dynamical Systems.

Embedding entropy: a nonlinear measure of dynamical causality

Quantile Transfer Entropy: Measuring the Heterogeneous Information Transfer of Nonlinear Time Series.

Detection Of Causality Between Process Variables Based On Industrial Alarm Data Using Transfer Entropy

Unraveling coupling delays through a transfer entropy analysis in stochastic processes and non-linear systems

Research and Application of Causal Network Modeling Based on Process Knowledge and Modified Transfer Entropy

A New Transfer Entropy Approach Based on Information Granulation and Clustering for Root Cause Analysis

Causation and Information Flow with Respect to Relative Entropy