Abstract:Boosting methods combine a set of moderately accurate weaklearners to form a highly accurate predictor. Despite the practical importance of multi-class boosting, it has received far less attention than its binary counterpart. In this work, we propose a fully-corrective multi-class boosting formulation which directly solves the multi-class problem without dividing it into multiple binary classification problems. In contrast, most previous multi-class boosting algorithms decompose a multi-boost problem into multiple binary boosting problems. By explicitly deriving the Lagrange dual of the primal optimization problem, we are able to construct a column generation-based fully-corrective approach to boosting which directly optimizes multi-class classification performance. The new approach not only updates all weak learners' coefficients at every iteration, but does so in a manner flexible enough to accommodate various loss functions and regularizations. For example, it enables us to introduce structural sparsity through mixed-norm regularization to promote group sparsity and feature sharing. Boosting with shared features is particularly beneficial in complex prediction problems where features can be expensive to compute. Our experiments on various data sets demonstrate that our direct multi-class boosting generalizes as well as, or better than, a range of competing multi-class boosting methods. The end result is a highly effective and compact ensemble classifier which can be trained in a distributed fashion.

Partially Corrective AdaBoost

AdaBoost Algorithm Using Multi-Step Correction

Boosting the Partial Least Square Algorithm for Regression Modelling

Multi-class AdaBoost ELM

Totally-Corrective Multi-Class Boosting

Totally Corrective Multiclass Boosting with Binary Weak Learners

A Revised AdaBoost Algorithm——AD AdaBoost

Totally Corrective Boosting for Regularized Risk Minimization

RotBoost: A technique for combining Rotation Forest and AdaBoost

A New Diverse AdaBoost Classifier

A Direct Formulation for Totally-Corrective Multi-Class Boosting

Z-Adaboost: Boosting 2-thresholded weak classifiers for object detection

Fully corrective boosting with arbitrary loss and regularization.

Two-Stage Multi-Class Adaboost for Facial Expression Recognition

Boosting Bootstrap Fld Subspaces In Classification

A Direct Approach to Multi-class Boosting and Extensions

The success of AdaBoost and its application in portfolio management

Asymmetric Totally-Corrective Boosting for Real-Time Object Detection

The Synergy Between PAV and AdaBoost

LACBoost and FisherBoost: Optimally Building Cascade Classifiers

A new boosting algorithm based on dual averaging scheme