A scalable direct formulation for multi-class boosting
Sakrapee Paisitkriangkrai,Chunhua Shen,Anton van den Hengel
2012-01-01
Abstract:We present a scalable and effective classification model for multi-class boosting classification. In [1], a direct formulation for multiclass boosting was introduced. Unlike many existing multi-class boosting algorithms, which rely on output coding matrices, the approach in [1] directly maximizes the multi-class margin. The major problem of [1] is its extremely high computational complexity, which hampers its application on real-world problems. In this work, we propose a scalable and simple stage-wise boosting method, which also directly maximizes the multiclass margin. Our approach has several advantages: (1) it is simple and computationally efficient to train. The approach can speed up the training time by more than two orders of magnitude without sacrificing the classification accuracy. (2) Like traditional AdaBoost, it is parameter free and empirically demonstrates excellent generalization performance. In contrast, one has to tune the regularization parameter for the multi-class boosting of [1]. Experimental results on challenging multiclass machine learning and vision tasks demonstrate that the proposed approach substantially improves the convergence rate and accuracy of the final visual detector at no additional computational cost. Our new formulation opens up a new way of incorporating prior knowledge into multi-class problems, e.g., feature sharing, hierarchical features, etc.