Abstract:Causality inference is prone to spurious causal interactions, due to the substantial confounders in a complex system. While many existing methods based on the statistical methods or dynamical methods attempt to address misidentification challenges, there remains a notable lack of effective methods to infer causality, in particular in the presence of invisible/unobservable confounders. As a result, accurately inferring causation with invisible confounders remains a largely unexplored and outstanding issue in data science and AI fields. In this work, we propose a method to overcome such challenges to infer dynamical causality under invisible confounders (CIC method) and further reconstruct the invisible confounders from time-series data by developing an orthogonal decomposition theorem in a delay embedding space. The core of our CIC method lies in its ability to decompose the observed variables not in their original space but in their delay embedding space into the common and private subspaces respectively, thereby quantifying causality between those variables both theoretically and computationally. This theoretical foundation ensures the causal detection for any high-dimensional system even with only two observed variables under many invisible confounders, which is actually a long-standing problem in the field. In addition to the invisible confounder problem, such a decomposition actually makes the intertwined variables separable in the embedding space, thus also solving the non-separability problem of causal inference. Extensive validation of the CIC method is carried out using various real datasets, and the experimental results demonstrates its effectiveness to reconstruct real biological networks even with unobserved confounders.

Hidden Variable Discovery Based on Regression and Entropy

A Versatile Causal Discovery Framework to Allow Causally-Related Hidden Variables

Applications of Common Entropy for Causal Inference

Dynamical causality under invisible confounders

DARING: Differentiable Causal Discovery with Residual Independence

Causal Discovery from Incomplete Data using An Encoder and Reinforcement Learning

Embedding entropy: a nonlinear measure of dynamical causality

Causal Feature Selection via Transfer Entropy

Causal Discovery in Linear Models with Unobserved Variables and Measurement Error

Direct Causality Detection Via the Transfer Entropy Approach

An Entropy Based Method for Causal Discovery in Linear Acyclic Model.

Differentiable Causal Discovery For Latent Hierarchical Causal Models

Causal Layering via Conditional Entropy

Causal Inference with Latent Variables: Recent Advances and Future Prospectives

Recursively Learning Causal Structures Using Regression-based Conditional Independence Test

Predictability Analysis of Regression Problems via Conditional Entropy Estimations

Variable Selection with Copula Entropy

Inferring Causal Directions from Uncertain Data

Data-driven Causal Inference Based on a Modified Transfer Entropy

Detection of Unobserved Common Causes based on NML Code in Discrete, Mixed, and Continuous Variables

Reinterpreting causal discovery as the task of predicting unobserved joint statistics