Abstract:Boosting methods combine a set of moderately accurate weaklearners to form a highly accurate predictor. Despite the practical importance of multi-class boosting, it has received far less attention than its binary counterpart. In this work, we propose a fully-corrective multi-class boosting formulation which directly solves the multi-class problem without dividing it into multiple binary classification problems. In contrast, most previous multi-class boosting algorithms decompose a multi-boost problem into multiple binary boosting problems. By explicitly deriving the Lagrange dual of the primal optimization problem, we are able to construct a column generation-based fully-corrective approach to boosting which directly optimizes multi-class classification performance. The new approach not only updates all weak learners' coefficients at every iteration, but does so in a manner flexible enough to accommodate various loss functions and regularizations. For example, it enables us to introduce structural sparsity through mixed-norm regularization to promote group sparsity and feature sharing. Boosting with shared features is particularly beneficial in complex prediction problems where features can be expensive to compute. Our experiments on various data sets demonstrate that our direct multi-class boosting generalizes as well as, or better than, a range of competing multi-class boosting methods. The end result is a highly effective and compact ensemble classifier which can be trained in a distributed fashion.

Multi-class Boosting Based on Phase-out Model

Max-margin Based Bayesian Classifier.

Probability Estimation for Multi-Class Classification Using AdaBoost

A Multi-Class Large Margin Classifier

Boosting Probabilistic Reasoning Model for Face Recognition

Multi-Classification using One-versus-One Deep Learning Strategy with Joint Probability Estimates

A scalable stage-wise approach to large-margin multi-class loss based boosting

Multi-Class Classification from Single-Class Data with Confidences

Neural network for multi-class classification by boosting composite stumps.

SMOTEMultiBoost: Leveraging the SMOTE with MultiBoost to Confront the Class Imbalance in Supervised Learning

Probabilistic Classification Vector Machine for Multi-Class Classification

A Direct Approach to Multi-class Boosting and Extensions

A scalable direct formulation for multi-class boosting

Multi-Class Imbalance Problem: A Multi-Objective Solution

Multi-Class Learning: from Theory to Algorithm.

Multi-Class L(2)-Boost With The Scoring Coding

Multi-class ℓ2-Boost with the scoring coding.

Multi-Class Imbalance Classification Based on Data Distribution and Adaptive Weights

A Boosting Approach to Exploit Instance Correlations for Multi-Instance Classification.

Multiclass Probabilistic Classification Vector Machine

Multiclass Classification Based On Meta Probability Codes