Abstract:Discovering the Bayesian network (BN) structure from big datasets containing rich causal relationships is becoming increasingly valuable for modeling and reasoning under uncertainties in many areas with big data gathered from sensors due to high volume and fast veracity. Most of the current BN structure learning algorithms have shortcomings facing big data. First, learning a BN structure from the entire big dataset is an expensive task which often ends in failure due to memory constraints. Second, it is quite difficult to select a learner from numerous BN structure learning algorithms to consistently achieve good learning accuracy. Lastly, there is a lack of an intelligent method that merges separately learned BN structures into a well structured BN network. To address these shortcomings, we introduce a novel parallel learning approach called PEnBayes (Parallel Ensemble-based Bayesian network learning). PEnBayes starts with an adaptive data preprocessing phase that calculates the Appropriate Learning Size and intelligently divides a big dataset for fast distributed local structure learning. Then, PEnBayes learns a collection of local BN Structures in parallel using a two-layered weighted adjacent matrix-based structure ensemble method. Lastly, PEnBayes merges the local BN Structures into a global network structure using the structure ensemble method at the global layer. For the experiment, we generate big data sets by simulating sensor data from patient monitoring, transportation, and disease diagnosis domains. The Experimental results show that PEnBayes achieves a significantly improved execution performance with more consistent and stable results compared with three baseline learning algorithms.

Data-Intensive Inferences Of Large-Scale Bayesian Networks

Data-Intensive Learning Of Uncertain Knowledge

Application of Bayesian Networks on Large-Scale Biological Data

Massively parallel learning of Bayesian networks with MapReduce for factor relationship analysis

Divide-and-Conquer Strategy for Large-Scale Dynamic Bayesian Network Structure Learning

Fast & Efficient Learning of Bayesian Networks from Data: Knowledge Discovery and Causality

Big Learning with Bayesian Methods

Reliable and Efficient Inference of Bayesian Networks from Sparse Data by Statistical Learning Theory

BN-Mapping:Visual Analysis of Geospatial Data with Bayesian Network

Learning Bayesian Network Structure from Large-Scale Datasets

BigBraveBN: algorithm of structural learning for bayesian networks with a large number of nodes

A Full Bayesian Approach to Sparse Network Inference Using Heterogeneous Datasets

PEnBayes: A Multi-Layered Ensemble Approach for Learning Bayesian Network Structure from Big Data

Faster-BNI: Fast Parallel Exact Inference on Bayesian Networks

Scaling Bayesian Network Parameter Learning with Expectation Maximization using MapReduce

An Empirical Study Of Massively Parallel Bayesian Networks Learning For Sentiment Extraction From Unstructured Text

Research on an advanced intelligence implementation system for engineering process in industrial field under big data

Scalable Community Detection in Massive Networks Using Aggregated Relational Data

Fast Parallel Exact Inference on Bayesian Networks: Poster

Uncertain Knowledge Representation And Inference For Lineage Processing Over Uncertain Data

LSBN: A Large-Scale Bayesian Structure Learning Framework for Model Averaging