Learning Discrete Latent Variable Structures with Tensor Rank Conditions

Zhengming Chen,Ruichu Cai,Feng Xie,Jie Qiao,Anpeng Wu,Zijian Li,Zhifeng Hao,Kun Zhang

2024-06-11

Abstract:Unobserved discrete data are ubiquitous in many scientific disciplines, and how to learn the causal structure of these latent variables is crucial for uncovering data patterns. Most studies focus on the linear latent variable model or impose strict constraints on latent structures, which fail to address cases in discrete data involving non-linear relationships or complex latent structures. To achieve this, we explore a tensor rank condition on contingency tables for an observed variable set $\mathbf{X}_p$, showing that the rank is determined by the minimum support of a specific conditional set (not necessary in $\mathbf{X}_p$) that d-separates all variables in $\mathbf{X}_p$. By this, one can locate the latent variable through probing the rank on different observed variables set, and further identify the latent causal structure under some structure assumptions. We present the corresponding identification algorithm and conduct simulated experiments to verify the effectiveness of our method. In general, our results elegantly extend the identification boundary for causal discovery with discrete latent variables and expand the application scope of causal discovery with latent variables.

Machine Learning

What problem does this paper attempt to address?

The paper focuses on developing a method to identify the causal structure of discrete latent variables in a model called the Discrete Latent Structure Model (Discrete LSM). The primary challenge addressed is the identification of the causal structure when dealing with discrete, unobserved (latent) variables, especially in cases where the data does not conform to linear relationships or has complex latent structures.

### Key Contributions

1. **Tensor Rank Condition**: The authors establish a connection between the tensor rank condition and the graphical patterns in a general discrete causal model. They show that the rank of a tensor representing the joint distribution of observed variables is related to the support of a variable that *d*-separates the observed variables. This connection helps in identifying the latent variables.

2. **Discrete Latent Structure Model (Discrete LSM)**: The paper defines a discrete LSM, which includes specific assumptions about the structure of the model, such as purity, the presence of at least 3 pure child variables for each latent variable, and sufficient observation assumptions.

3. **Identification of Measurement and Structure Models**: The authors propose an algorithm to identify both the measurement model (which determines the latent variables) and the structure model (which identifies the causal structure among the latent variables). The measurement model is fully identifiable, while the structure model is identified up to a Markov