Abstract:Discovering the Bayesian network (BN) structure from big datasets containing rich causal relationships is becoming increasingly valuable for modeling and reasoning under uncertainties in many areas with big data gathered from sensors due to high volume and fast veracity. Most of the current BN structure learning algorithms have shortcomings facing big data. First, learning a BN structure from the entire big dataset is an expensive task which often ends in failure due to memory constraints. Second, it is quite difficult to select a learner from numerous BN structure learning algorithms to consistently achieve good learning accuracy. Lastly, there is a lack of an intelligent method that merges separately learned BN structures into a well structured BN network. To address these shortcomings, we introduce a novel parallel learning approach called PEnBayes (Parallel Ensemble-based Bayesian network learning). PEnBayes starts with an adaptive data preprocessing phase that calculates the Appropriate Learning Size and intelligently divides a big dataset for fast distributed local structure learning. Then, PEnBayes learns a collection of local BN Structures in parallel using a two-layered weighted adjacent matrix-based structure ensemble method. Lastly, PEnBayes merges the local BN Structures into a global network structure using the structure ensemble method at the global layer. For the experiment, we generate big data sets by simulating sensor data from patient monitoring, transportation, and disease diagnosis domains. The Experimental results show that PEnBayes achieves a significantly improved execution performance with more consistent and stable results compared with three baseline learning algorithms.

Parallel Learning of Bayesian Networks Based on Ordering of Sets

Structure Learning Based on Ordering of Sets

Grouping Parallel Bayesian Network Structure Learning Algorithm Based on Variable Ordering

Learning directed acyclic graphs based on sparsest permutations

Improved Population-Based Incremental Learning of Bayesian Networks with partly known structure and parallel computing

A Parallel Algorithm for Learning Bayesian Networks

Hybrid Optimized Algorithm for Learning Bayesian Network Structure

A Parallel Algorithm for Exact Bayesian Structure Discovery in Bayesian Networks

Learning Bayesian Networks Using a Parallel EM Approach

Efficient Learning Bayesian Networks Using Pso

A Novel Parallel Framework for Pursuit Learning Schemes

Parallel Module Network Learning on Distributed Memory Multiprocessors

Efficient Heuristics for Learning Bayesian Network from Labeled and Unlabeled Data

Learning the structure of Bayesian networks with ancestral and/or heuristic partition

Parallel structural learning of Bayesian networks: Iterative divide and conquer algorithm based on structural fusion

A Ring-Based Distributed Algorithm for Learning High-Dimensional Bayesian Networks

End-to-End Bayesian Networks Exact Learning in Shared Memory

Improved Bayesian Network Structure Learning With Node Ordering Via K2 Algorithm

The impact of variable ordering on Bayesian network structure learning

Hybrid Incremental Learning Algorithms for Bayesian Network Structures

PEnBayes: A Multi-Layered Ensemble Approach for Learning Bayesian Network Structure from Big Data