LOCAL: Learning with Orientation Matrix to Infer Causal Structure from Time Series Data

Yue Cheng,Jiajun Zhang,Weiwei Xing,Xiaoyu Guo,Xiaohui Gao
2024-10-28
Abstract:Discovering the underlying Directed Acyclic Graph (DAG) from time series observational data is highly challenging due to the dynamic nature and complex nonlinear interactions between variables. Existing methods often struggle with inefficiency and the handling of high-dimensional data. To address these research gap, we propose LOCAL, a highly efficient, easy-to-implement, and constraint-free method for recovering dynamic causal structures. LOCAL is the first attempt to formulate a quasi-maximum likelihood-based score function for learning the dynamic DAG equivalent to the ground truth. On this basis, we propose two adaptive modules for enhancing the algebraic characterization of acyclicity with new capabilities: Asymptotic Causal Mask Learning (ACML) and Dynamic Graph Parameter Learning (DGPL). ACML generates causal masks using learnable priority vectors and the Gumbel-Sigmoid function, ensuring the creation of DAGs while optimizing computational efficiency. DGPL transforms causal learning into decomposed matrix products, capturing the dynamic causal structure of high-dimensional data and enhancing interpretability. Extensive experiments on synthetic and real-world datasets demonstrate that LOCAL significantly outperforms existing methods, and highlight LOCAL's potential as a robust and efficient method for dynamic causal discovery. Our code will be available soon.
Machine Learning,Artificial Intelligence
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to solve the highly challenging problem of discovering latent directed acyclic graphs (DAGs) from time - series observational data. Specifically, the authors propose improvements in the following aspects: 1. **Dynamic systems and complex nonlinear interactions**: - Time - series data has dynamic characteristics, and the complex nonlinear interactions between variables make the inference of causal structures very difficult. - Existing methods are often inefficient when dealing with high - dimensional data and have difficulty handling complex nonlinear relationships. 2. **Limitations of existing methods**: - Many existing methods require a large amount of computational resources during the optimization process and have difficulty handling high - dimensional dynamic systems. - For example, the DYNOTEARS algorithm and its extended methods require an extremely long time for optimization in high - dimensional cases, even though they enhance identifiability through intervention data. 3. **The need for an efficient, easy - to - implement, and unconstrained method**: - A surrogate model that can quickly infer dynamic causal structures in high - dimensional cases is needed to overcome the deficiencies of existing methods. To solve these problems, the authors propose a new framework named LOCAL, with the following main features: - **Quasi - maximum likelihood score function**: It is the first attempt to use the quasi - maximum likelihood score function to learn the dynamic causal structure equivalent to the true DAG. - **Adaptive module - enhanced algebraic representation**: Two adaptive modules - Asymptotic Causal Mask Learning (ACML) and Dynamic Graph Parameter Learning (DGPL) are proposed to enhance the algebraic representation ability of acyclicity. - **ACML**: A causal mask is generated using a learnable priority vector and the Gumbel - Sigmoid function to ensure the creation of a DAG while optimizing computational efficiency. - **DGPL**: Causal learning is transformed into a decomposed matrix product to capture the dynamic causal structure of high - dimensional data and improve interpretability. Through these innovations, LOCAL can infer dynamic causal structures in high - dimensional time - series data more accurately and efficiently. ### Formula presentation To ensure the correctness and readability of the formulas, the following are the key formulas involved in the paper: 1. **Definition of Dynamic Bayesian Network (DBN)**: \[ \mathbf{X}_t=\mathbf{X}_t\mathbf{A}+\mathbf{C}\mathbf{B}+\mathbf{E} \] where $\mathbf{X}_t$ is the observational data at time step $t$, $\mathbf{A}$ is the instantaneous dependence matrix, $\mathbf{B}$ is the lag - dependence matrix, and $\mathbf{C}$ is the exogenous noise variable. 2. **Quasi - maximum likelihood objective function**: \[ S(\mathbf{A},\mathbf{B};\mathbf{X}) = L(\mathbf{A},\mathbf{B};\mathbf{X})+\lambda_1h(\mathbf{A})+\lambda_2(R_{\ell_1}(\mathbf{B})+R_{\ell_1}(\mathbf{A})) \] where $L(\mathbf{A},\mathbf{B};\mathbf{X})$ is the quasi - maximum likelihood estimation (QMLE), $h(\mathbf{A})$ is the DAG constraint, and $\lambda_1$ and $\lambda_2$ are regularization parameters. 3. **Asymptotic Causal Mask Learning (ACML)**: \[ \mathbf{O}_{\tau,\epsilon}(\mathbf{p})_{ij}=\sigma_{\epsilon}(\mathbf{p}_j - \mathbf{p}_i) \] where $\sigma_{\epsilon}(x)$ is the Gumbel - Sigmoid function centered at $\epsilon$. 4. **Dynamic graph parameter learning**