Abstract:In Bayesian network structure learning, the quality of the directed graph learned by the constraint-based approaches can be greatly affected by the order of choosing variable pairs and the order of selecting condition sets for testing conditional independence. Inspired by the strong connection between the degree of mutual information shared by two variables and their conditional independence, we introduce the M-ordering concept, where a matrix is precomputed from the observational data with variables ordered increasingly by their respective degree of mutual information with the target variable under concern. Given the M-ordering matrix, we propose a strategy called Weakest Mutual-Information-First Strategy (WMIF), which is integrated into the PC-algorithm in two aspects: an MI-based edge removal strategy, and an MI-based condition set generation strategy. The MI-based edge removal strategy is to always select the variable pair with the weakest mutual information to test their conditional independence; the condition set generation strategy is to construct a conditioning set where variables bearing a weaker degree of mutual information with the target variable are always considered first. We prove that the weakest MI-based edge removal strategy is sound, and our PC-MI algorithm, a PC variant empowered by the WMIF strategy, is order-independent. Moreover, in PC algorithms, the number of conditional independence tests increases exponentially with the number of random variables; we show that the WMIF strategy can effectively reduce the complexity (bounded by o ( | V | ( 2 | a d j ( X ) | − | V | 2 2 ) )). We have conducted experiments with both low-dimensional and high-dimensional data sets, and the results indicate that PC-MI outperforms the state-of-the-art approaches. More importantly, the order-agnostic property of PC-MI can be extremely useful when it is hard to prescribe a meaningful variable ordering as needed in some other PC algorithms.

Flexible model weighting for one-dependence estimators based on point-wise independence analysis

Model Weighting for One-Dependence Estimators by Measuring the Independence Assumptions

Alleviating the independence assumptions of averaged one-dependence estimators by model weighting

Alleviating the Attribute Conditional Independence and I.I.D. Assumptions of Averaged One-Dependence Estimator by Double Weighting

Self-Adaptive Attribute Value Weighting for Averaged One-Dependence Estimators.

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Instance-based weighting filter for superparent one-dependence estimators

Attribute Value Weighted Average of One-Dependence Estimators

Attribute Value Weighted Averaged One-Dependence Estimators with Kullback–Leibler Divergence

Averaged Tree-Augmented One-Dependence Estimators

Semi-supervised Weighting for Averaged One-Dependence Estimators

A Mathematical Framework to Characterize the Dependency Structures in Multimodal Learning with Minimax Principle

Identification of Informational and Probabilistic Independence by Adaptive Thresholding

Combination Weighting Approach Based on Deviation Function and Joint-Entropy

On the Effect of Suboptimal Estimation of Mutual Information in Feature Selection and Classification

Learning Bayesian Network Structures Using Weakest Mutual-Information-first Strategy

Independence test via mutual information in the presence of measurement errors

Weighted One-Dependence Forests Classifier.

A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization

Exploiting the Implicit Independence Assumption for Learning Directed Graphical Models.

A Mathematical Framework for Characterizing Dependency Structures of Multimodal Learning