Abstract:Sparse probabilistic graphical models play an important role in structured prediction when the dependency structure is unknown. By inducing sparsity over edge parameters, a typical sparse graphical model can combine structure learning and parameter estimation under a unified optimization framework. In this thesis, we propose three specific sparse graphical models accompanied by their applications in data restoration and multi-label classification respectively. For the data restoration task, we propose random mixed field (RMF) model to explore mixed-attribute correlations among data. The RMF model is capable of handling mixed-attribute data denoising and imputation simultaneously. Meanwhile, RMF employs a structured mean-field variational approach to decouple continuous-discrete interactions to achieve approximate inference. The effectiveness of this model is evaluated on both synthetic and real-world data. For the multi-label classification task, we propose correlated logistic model (CorrLog) and conditional graphical lasso (CGL), to learn conditional label correlations. (1) The CorrLog model characterizes pairwise label correlations via scalar parameters, thus effects in an explicit (or direct) fashion. More specifically, CorrLog extends conventional logistic regression by jointly modelling label correlations. In addition, elastic-net regularization is employed to induce sparsity over the scalar parameters that define label correlations. CorrLog can be efficiently learned by regularized maximum pseudo likelihood estimation which enjoys a satisfying generalization bound. Besides, message passing algorithm is applied to solve the multi-label prediction problem. (2) The CGL model further leverages features in modelling pairwise label correlations in terms of parametric functions of the input features, which effects in an implicit (or indirect) fashion. In general, CGL provides a unified Bayesian framework for structure and parameter learning conditioned on input features. We formulate the multi-label prediction as CGL inference problem, which is solved by a mean field variational approach. Meanwhile, CGL learning is efficient after applying the maximum a posterior (MAP) methodology and solved by a proximal gradient procedure. The effectiveness of CorrLog and CGL are evaluated on several benchmark multi-label classification datasets.

Joint Maximum Margin and Maximum Entropy Learning of Graphical Models

Partially Observed Maximum Entropy Discrimination Markov Networks

Laplace Maximum Margin Markov Networks.

Large Margin Boltzmann Machines and Large Margin Sigmoid Belief Networks

Large Margin Learning of Upstream Scene Understanding Models

Learning Sparse Graphical Models for Data Restoration and Multi-Label Classification

Marginal Pseudo-Likelihood Learning of Markov Network structures

Gibbs Max-margin Topic Models with Data Augmentation

High-dimensional Mixed Graphical Models

Learning to Solve the Constrained Most Probable Explanation Task in Probabilistic Graphical Models

Exact Maximum Margin Structure Learning of Bayesian Networks

Graph Learning for Latent-Variable Gaussian Graphical Models under Laplacian Constraints

Large Discriminative Structured Set Prediction Modeling With Max-Margin Markov Network For Lossless Image Coding

Graph-based Consensus Maximization among Multiple Supervised and Unsupervised Models

Multimodal Understanding Through Correlation Maximization and Minimization

Structure Learning of Contextual Markov Networks using Marginal Pseudo-likelihood

Scalable Inference in Max-Margin Topic Models

Distributionally Robust Graphical Models

Learning Undirected Graphical Models with Structure Penalty

Graph Structure Learning with Interpretable Bayesian Neural Networks

Scene understanding with discriminative structured prediction