Abstract:Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. However, the resulting learning problems are usually hard to solve because of the non-smoothness of the margin loss. Existing approaches to building max-margin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional mean-field assumptions on the desired posterior distributions. This paper presents an alternative approach by defining a new max-margin loss. Namely, we present Gibbs max-margin supervised topic models, a latent variable Gibbs classifier to discover hidden topic representations for various tasks, including classification, regression and multi-task learning. Gibbs max-margin supervised topic models minimize an expected margin loss, which is an upper bound of the existing margin loss derived from an expected prediction rule. By introducing augmented variables and integrating out the Dirichlet variables analytically by conjugacy, we develop simple Gibbs sampling algorithms with no restrictive assumptions and no need to solve SVM subproblems. Furthermore, each step of the "augment-and-collapse" Gibbs sampling algorithms has an analytical conditional distribution, from which samples can be easily drawn. Experimental results on several medium-sized and large-scale data sets demonstrate significant improvements on time effciency. The classification performance is also improved over competitors on binary, multi-class and multi-label classification tasks.

Bayesian Max-margin Multi-Task Learning with Data Augmentation.

Nonparametric Bayesian Multi-task Learning with Max-margin Posterior Regularization

Max-margin Basedbayesian Classifier

Nonparametric Bayesian Multi-Task Large-margin Classification.

Gibbs Max-margin Topic Models with Data Augmentation

Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning

Label-Sensitive Task Grouping by Bayesian Nonparametric Approach for Multi-Task Multi-Label Learning.

Fast Max-Margin Matrix Factorization with Data Augmentation.

A Good Data Augmentation Policy Is Not All You Need: A Multi-Task Learning Perspective

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Cross-Modal Data Augmentation for Tasks of Different Modalities

Bayesian inference for transductive learning of kernel matrix using the Tanner-Wong data augmentation algorithm

Bayesian Maximum Margin Principal Component Analysis

Bayesian task embedding for few-shot Bayesian optimization

Multi-task Model and Feature Joint Learning

Adaptive and Robust Multi-Task Learning

Improved Bayesian Logistic Supervised Topic Models with Data Augmentation.

A Bayesian approach to multi-task learning with network lasso

On Data Augmentation for Extreme Multi-label Classification

Bayesian learning of feature spaces for multitask regression

Bayesian learning of feature spaces for multitasks problems