Abstract:Boosting methods combine a set of moderately accurate weaklearners to form a highly accurate predictor. Despite the practical importance of multi-class boosting, it has received far less attention than its binary counterpart. In this work, we propose a fully-corrective multi-class boosting formulation which directly solves the multi-class problem without dividing it into multiple binary classification problems. In contrast, most previous multi-class boosting algorithms decompose a multi-boost problem into multiple binary boosting problems. By explicitly deriving the Lagrange dual of the primal optimization problem, we are able to construct a column generation-based fully-corrective approach to boosting which directly optimizes multi-class classification performance. The new approach not only updates all weak learners' coefficients at every iteration, but does so in a manner flexible enough to accommodate various loss functions and regularizations. For example, it enables us to introduce structural sparsity through mixed-norm regularization to promote group sparsity and feature sharing. Boosting with shared features is particularly beneficial in complex prediction problems where features can be expensive to compute. Our experiments on various data sets demonstrate that our direct multi-class boosting generalizes as well as, or better than, a range of competing multi-class boosting methods. The end result is a highly effective and compact ensemble classifier which can be trained in a distributed fashion.

Sharing Features in Multi-Class Boosting Via Group Sparsity

Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

A Direct Approach to Multi-class Boosting and Extensions

Multi-View Correlated Feature Learning by Uncovering Shared Component.

A scalable stage-wise approach to large-margin multi-class loss based boosting

A scalable direct formulation for multi-class boosting

Multi-class AdaBoost ELM

Totally-Corrective Multi-Class Boosting

Adaptive Object Detection by Implicit Sub-Class Sharing Features

A Direct Formulation for Totally-Corrective Multi-Class Boosting

Totally Corrective Multiclass Boosting with Binary Weak Learners

Fully corrective boosting with arbitrary loss and regularization.

Adaptive Sharing for Image Classification.

A Scalable Stagewise Approach to Large-Margin Multiclass Loss-Based Boosting

Two-Stage Multi-Class Adaboost for Facial Expression Recognition

Totally Corrective Boosting for Regularized Risk Minimization

Implicit JointBoost for Multiclass Object Detection under High Intra-Class Variation

RandomBoost: Simplified Multiclass Boosting Through Randomization

Multi-fairness under class-imbalance

Boosting with structural sparsity: A differential inclusion approach

Learning Cascaded Shared-Boost Classifiers for Part-Based Object Detection