Abstract:This study proposes the first Bayesian approach for learning high-dimensional linear Bayesian networks. The proposed approach iteratively estimates each element of the topological ordering from backward and its parent using the inverse of a partial covariance matrix. The proposed method successfully recovers the underlying structure when Bayesian regularization for the inverse covariance matrix with unequal shrinkage is applied. Specifically, it shows that the number of samples $n = \Omega( d_M^2 \log p)$ and $n = \Omega(d_M^2 p^{2/m})$ are sufficient for the proposed algorithm to learn linear Bayesian networks with sub-Gaussian and 4m-th bounded-moment error distributions, respectively, where $p$ is the number of nodes and $d_M$ is the maximum degree of the moralized graph. The theoretical findings are supported by extensive simulation studies including real data analysis. Furthermore the proposed method is demonstrated to outperform state-of-the-art frequentist approaches, such as the BHLSM, LISTEN, and TD algorithms in synthetic data.

What problem does this paper attempt to address?

This paper proposes a new Bayesian method for learning high-dimensional linear Bayesian networks (linear structural equation models). The current methods are mainly divided into three categories: likelihood-based methods, inverse covariance matrix-based methods, and node regression-based methods. However, there is relatively little research under the Bayesian framework due to high computational costs and the immaturity of the field. The main objective of the paper is to develop an iterative algorithm that uses the inverse covariance matrix to estimate the topological order and parent nodes of each node, and applies Bayesian regularization with unequal shrinkage (BAGUS). The theoretical analysis shows that for linear Bayesian networks with sub-Gaussian and 4th-order moment bounded error distributions, the algorithm can recover the underlying structure when the sample size reaches n=Ω(d^2_M log p) and n=Ω(d^2_M p^2/m), respectively, where d_M is the maximum degree of the moralized graph and p is the number of nodes. Through extensive simulation studies, the paper confirms the theoretical findings of the algorithm and compares it with the current state-of-the-art frequentist methods such as BHLSM, LISTEN, TD, and US algorithms, demonstrating its superiority in synthetic data. In addition, the paper demonstrates the applicability of the algorithm in estimating the sales relationship of various products through online shopping center order amount data. The structure of the paper includes background introduction, algorithm description, theoretical guarantees, comparison with other algorithms, simulation experiment evaluation, practical application examples, and discussion and future work recommendations. In summary, this paper aims to solve the structural learning problem of high-dimensional linear Bayesian networks using Bayesian methods and provides an efficient and theoretically supported algorithm.

Bayesian Approach to Linear Bayesian Networks

Scalable Bayesian High-dimensional Local Dependence Learning

High Dimensional Bayesian Network Classification with Network Global-Local Shrinkage Priors

Bayes optimal learning in high-dimensional linear regression with network side information

Learning Gaussian Networks

A Full Bayesian Approach to Sparse Network Inference Using Heterogeneous Datasets

High-Dimensional Bayesian Network Inference From Systems Genetics Data Using Genetic Node Ordering

Bayesian Inference with Deep Weakly Nonlinear Networks

Bayesian learning of multiple directed networks from observational data

Reliable and Efficient Inference of Bayesian Networks from Sparse Data by Statistical Learning Theory

Learning Moral Graphs in Construction of High-Dimensional Bayesian Networks for Mixed Data

An Asymptotically Optimal Coordinate Descent Algorithm for Learning Bayesian Networks from Gaussian Models

Scalable Bayesian regression in high dimensions with multiple data sources

Bayesian multiple Gaussian graphical models for multilevel variables from unknown classes

Bayesian Model Selection in High-Dimensional Settings

Local Structure Discovery in Bayesian Networks

Bayesian Learning of Relational Graph in Semiparametric High-dimensional Time Series

Bayesian Model Selection for High-Dimensional Ising Models, With Applications to Educational Data

Bayesian Analysis for Over-parameterized Linear Model without Sparsity

A modeling framework for detecting and leveraging node-level information in Bayesian network inference