Mutual-information-inspired heuristics for constraint-based causal structure learning

Xiaolong Qi,Xiaocong Fan,Huiling Wang,Ling Lin,Yang Gao
DOI: https://doi.org/10.1016/j.ins.2020.12.009
IF: 8.1
2021-06-01
Information Sciences
Abstract:<p>In constraint-based approaches to Bayesian network structure learning, when the assumption of orientation-faithfulness is violated, not only the correctness of edge orientation can be greatly degraded, the soaring cost of conditional independence testing also limits their applicability in learning very large causal networks. Inspired by the strong connection between the degree of mutual information shared by two variables and their conditional independence, we extend the PC-MI algorithm in two ways: (a) the Weakest Edge-First (WEF) strategy implemented in PC-MI is further integrated with Markov-chain consistency to reduce the number of independence testing and sustain the number of false positive edges in skeletal learning; (b) the Smaller Adjacency-Set (SAS) strategy is proposed and we prove that the Smaller Adjacency-Set captures sufficient information for determining whether an unshielded triple forms a v-structure. We have conducted experiments with both low-dimensional and high-dimensional data sets, and the results indicate that our MIIPC approach outperforms the state-of-the-art approaches in both the quality of learning and the execution time.</p>
computer science, information systems
What problem does this paper attempt to address?
The main goal of this paper is to propose an effective strategy for learning high-quality Bayesian network structures in cases where the orientation faithfulness assumption is partially violated. Specifically, the paper attempts to address the following issues: 1. **Reducing the number of conditional independence tests**: In constraint-based methods, when the orientation faithfulness assumption is violated, it not only affects the correctness of edge directions but also significantly increases the cost of conditional independence tests, thereby limiting its application in large-scale causal networks. Therefore, the paper proposes a method that combines the Weakest Edge First (WEF) strategy with Markov chain consistency to reduce the number of conditional independence tests and control the number of false positive edges in skeleton learning. 2. **Improving V-structure identification**: The paper proposes a new heuristic algorithm called "Smaller Adjacency Set" (SAS) and demonstrates that it can capture sufficient information to determine whether an unshielded triple forms a V-structure. This method helps improve the accuracy of V-structure identification. 3. **Enhancing learning quality and execution efficiency**: By integrating the above two strategies (WEF and SAS), the proposed MIIPC algorithm outperforms existing state-of-the-art methods in terms of both the quality of causal structure learning and execution time. In summary, the paper aims to improve constraint-based causal structure learning methods by introducing new heuristic strategies, maintaining high learning quality, and reducing computational costs when handling high-dimensional datasets.