Abstract:Undirected graphical models known as Markov networks are popular for a wide variety of applications ranging from statistical physics to computational biology. Traditionally, learning of the network structure has been done under the assumption of chordality which ensures that efficient scoring methods can be used. In general, non-chordal graphs have intractable normalizing constants which renders the calculation of Bayesian and other scores difficult beyond very small-scale systems. Recently, there has been a surge of interest towards the use of regularized pseudo-likelihood methods for structural learning of large-scale Markov network models, as such an approach avoids the assumption of chordality. The currently available methods typically necessitate the use of a tuning parameter to adapt the level of regularization for a particular dataset, which can be optimized for example by cross-validation. Here we introduce a Bayesian version of pseudo-likelihood scoring of Markov networks, which enables an automatic regularization through marginalization over the nuisance parameters in the model. We prove consistency of the resulting MPL estimator for the network structure via comparison with the pseudo information criterion. Identification of the MPL-optimal network on a prescanned graph space is considered with both greedy hill climbing and exact pseudo-Boolean optimization algorithms. We find that for reasonable sample sizes the hill climbing approach most often identifies networks that are at a negligible distance from the restricted global optimum. Using synthetic and existing benchmark networks, the marginal pseudo-likelihood method is shown to generally perform favorably against recent popular inference methods for Markov networks.

Partially Observed Maximum Entropy Discrimination Markov Networks

Laplace Maximum Margin Markov Networks.

Joint Maximum Margin and Maximum Entropy Learning of Graphical Models

Learning Bayesian Networks Using the Constrained Maximum a Posteriori Probability Method

Marginal Pseudo-Likelihood Learning of Markov Network structures

Primal Sparse Max-margin Markov Networks

Struct-MMSB: Mixed Membership Stochastic Blockmodels with Interpretable Structured Priors

Structure Learning of Contextual Markov Networks using Marginal Pseudo-likelihood

Max-Margin Nonparametric Latent Feature Models for Link Prediction.

MedLDA: maximum margin supervised topic models

On Primal and Dual Sparsity of Markov Networks.

Discrimination Reveals Reconstructability of Multiplex Networks from Partial Observations

Learning Mixture Models with the Regularized Latent Maximum Entropy Principle

Max-margin Latent Feature Relational Models for Entity-Attribute Networks

Multimodal Poisson Gamma Belief Network.

Infinite max-margin factor analysis via data augmentation

Nonparametric Bayesian Multi-task Learning with Max-margin Posterior Regularization

Structural Minimax Probability Machine

Semi-supervised Multi-View Maximum Entropy Discrimination with Expectation Laplacian Regularization

Exact Maximum Margin Structure Learning of Bayesian Networks

Chunking with Max-Margin Markov Networks