Abstract:Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. However, the resulting learning problems are usually hard to solve because of the non-smoothness of the margin loss. Existing approaches to building max-margin supervised topic models rely on an iterative procedure to solve multiple latent SVM subproblems with additional mean-field assumptions on the desired posterior distributions. This paper presents an alternative approach by defining a new max-margin loss. Namely, we present Gibbs max-margin supervised topic models, a latent variable Gibbs classifier to discover hidden topic representations for various tasks, including classification, regression and multi-task learning. Gibbs max-margin supervised topic models minimize an expected margin loss, which is an upper bound of the existing margin loss derived from an expected prediction rule. By introducing augmented variables and integrating out the Dirichlet variables analytically by conjugacy, we develop simple Gibbs sampling algorithms with no restrictive assumptions and no need to solve SVM subproblems. Furthermore, each step of the "augment-and-collapse" Gibbs sampling algorithms has an analytical conditional distribution, from which samples can be easily drawn. Experimental results on several medium-sized and large-scale data sets demonstrate significant improvements on time effciency. The classification performance is also improved over competitors on binary, multi-class and multi-label classification tasks.

Large Margin Learning of Upstream Scene Understanding Models

Scalable Inference in Max-Margin Topic Models

Joint Maximum Margin and Maximum Entropy Learning of Graphical Models

Online Bayesian Max-Margin Subspace Multi-View Learning.

MedLDA: maximum margin supervised topic models

Gibbs Max-margin Topic Models with Data Augmentation

Large Margin Boltzmann Machines and Large Margin Sigmoid Belief Networks

Online Bayesian Max-Margin Subspace Learning for Multi-View Classification and Regression

Monte Carlo Methods for Maximum Margin Supervised Topic Models.

Large-Margin Predictive Latent Subspace Learning for Multiview Data Analysis.

Predictive Subspace Learning for Multi-view Data: a Large Margin Approach.

Gibbs Max-Margin Topic Models with Fast Sampling Algorithms.

Max-Margin Deep Diverse Latent Dirichlet Allocation With Continual Learning

MedLDA: A General Framework of Maximum Margin Supervised Topic Models

Soft Margin Consistency Based Scalable Multi-View Maximum Entropy Discrimination

Scene understanding with discriminative structured prediction

MMH: Maximum Margin Supervised Harmoniums

Semisupervised Local Preserving Embedding Algorithm Based on Maximum Margin Criterion for Large-Scale Data Streams.

Multi-modal Max-Margin Supervised Topic Model for Social Event Analysis

OCC-MLLM-Alpha:Empowering Multi-modal Large Language Model for the Understanding of Occluded Objects with Self-Supervised Test-Time Learning

Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion