Abstract:It is unpractical to learn the optimal structure of a big Bayesian network (BN) by exhausting the feasible structures, since the number of feasible structures is super exponential on the number of nodes. This paper proposes an approach to layer nodes of a BN by using the conditional independence testing. The parents of a node layer only belong to the layer, or layers who have priority over the layer. When a set of nodes has been layered, the number of feasible structures over the nodes can be remarkably reduced, which makes it possible to learn optimal BN structures for bigger sizes of nodes by accurate algorithms. Integrating the dynamic programming (DP) algorithm with the layering approach, we propose a hybrid algorithm — layered optimal learning (LOL) to learn BN structures. Benefitted by the layering approach, the complexity of the DP algorithm reduces to O(ρ2 n−1 ) from O(n2 n−1 ), where ρ < n. Meanwhile, the memory requirements for storing intermediate results are limited to O(C k#/2 <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k#</inf> ) from O(C n/2 <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</inf> ), where k # <n. A case study on learning a standard BN with 50 nodes is conducted. The results demonstrate the superiority of the LOL algorithm, with respect to the Bayesian information criterion (BIC) score criterion, over the hill-climbing, max-min hill-climbing, PC, and three-phrase dependency analysis algorithms.

Layer adaptive node selection in Bayesian neural networks: Statistical guarantees and implementation details

Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networks

Neuronized Priors for Bayesian Sparse Linear Regression

Sparse Bayesian Neural Networks: Bridging Model and Parameter Uncertainty through Scalable Variational Inference

High Dimensional Bayesian Network Classification with Network Global-Local Shrinkage Priors

Finding optimal Bayesian networks by a layered learning method

Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihood

A modeling framework for detecting and leveraging node-level information in Bayesian network inference

Reliable and Efficient Inference of Bayesian Networks from Sparse Data by Statistical Learning Theory

Impact of Parameter Sparsity on Stochastic Gradient MCMC Methods for Bayesian Deep Learning

Restricted Bayesian Neural Network

Incorporating Path Constraints into Bayesian Network under the Sparse Parent Graph

Efficient Model Compression for Bayesian Neural Networks

Misclassification bounds for PAC-Bayesian sparse deep learning

Deep Network Regularization via Bayesian Inference of Synaptic Connectivity

Sparsity-Aware Optimization of In-Memory Bayesian Binary Neural Network Accelerators

Bayesian sparsification for deep neural networks with Bayesian model reduction

Hierarchical-Bayesian-Based Sparse Stochastic Configuration Networks for Construction of Prediction Intervals

Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling

Variational Bayes Neural Network: Posterior Consistency, Classification Accuracy and Computational Challenges

From Bayesian Sparsity to Gated Recurrent Nets