Abstract:Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. However, the resulting learning problems are usually hard to solve because of the non-smoothness of the margin loss. Existing approaches to building max-margin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional mean-field assumptions on the desired posterior distributions. This paper presents an alternative approach by defining a new max-margin loss. Namely, we present Gibbs max-margin supervised topic models, a latent variable Gibbs classifier to discover hidden topic representations for various tasks, including classification, regression and multi-task learning. Gibbs max-margin supervised topic models minimize an expected margin loss, which is an upper bound of the existing margin loss derived from an expected prediction rule. By introducing augmented variables and integrating out the Dirichlet variables analytically by conjugacy, we develop simple Gibbs sampling algorithms with no restrictive assumptions and no need to solve SVM subproblems. Furthermore, each step of the "augment-and-collapse" Gibbs sampling algorithms has an analytical conditional distribution, from which samples can be easily drawn. Experimental results on several medium-sized and large-scale data sets demonstrate significant improvements on time effciency. The classification performance is also improved over competitors on binary, multi-class and multi-label classification tasks.

Max-margin Based Bayesian Classifier.

A Multi-Class Large Margin Classifier

Nonlinear Online Classification Algorithm with Probability Margin

Exact Maximum Margin Structure Learning of Bayesian Networks

Universality of max-margin classifiers

MassBayes: A New Generative Classifier with Multi-dimensional Likelihood Estimation.

A scalable stage-wise approach to large-margin multi-class loss based boosting

Bayesian Maximum Margin Principal Component Analysis

On the generalization of bayesian deep nets for multi-class classification

Bayesian Max-margin Multi-Task Learning with Data Augmentation.

Nonparametric Bayesian Multi-task Learning with Max-margin Posterior Regularization

Optimal Minimal Margin Maximization with Boosting

Local Probabilistic Model for Bayesian Classification: a Generalized Local Classification Model

Boosting Through Optimization of Margin Distributions

Optimal-margin evolutionary classifier

A scalable direct formulation for multi-class boosting

Large Margin Discriminative Loss for Classification

Nonparametric Bayesian Multi-Task Large-margin Classification.

Multi-classification Algorithm Based on Truncated Gaussian Prior and Variational Bayesian

Gibbs Max-margin Topic Models with Data Augmentation

A Scalable Stagewise Approach to Large-Margin Multiclass Loss-Based Boosting