Abstract:To address the imbalanced data problem in classification, the studies of the combination of AdaBoost, short for “Adaptive Boosting,” and cost-sensitive learning have shown convincing results in the literature. The cost-sensitive AdaBoost algorithms are practical since the “boosting” property in AdaBoost can iteratively enhance the small class of the cost-sensitive learning to solve the imbalanced data issue. However, the most available cost-sensitive AdaBoost algorithms are heuristic approaches, which are improved from the standard AdaBoost algorithm by cost-sensitively adjusting the voting weight parameters of weak classifiers or the sample updating weight parameters without strict theoretic proof. The algorithms are appended the cost-sensitive factors to focus on the high-cost and small-class samples, but they have no procedures to show the best place to add the cost factors and the cost factor value set. To complete the cost-sensitive AdaBoost algorithms’ framework, the present article has two main contributions. First, we summarize the popular cost-sensitive boosting algorithms in the literature and propose a generally comprehensive form. We name our specific one, the “AdaImC algorithm,” which is typically appliable to solve the imbalanced data classification problem with theoretic proof. Second, a statistical approach to prove the AdaImC algorithm is proposed to verify the inner relationship between the cost parameters. We show that our proposed algorithm in the machine learning field is identical to the Product of Experts (PoE) model in the statistics field. Besides, a way to determine the cost parameter value by the statistical analysis is introduced. Several numeric studies are listed finally to support our proposed algorithm.

A method of classification accuracy calculation for cost sensitive algorithms

A Method of Determining the Cost Weight with High-Dimensional Fitting

Learning Misclassification Costs for Imbalanced Classification on Gene Expression Data

Cost Sensitive Support Vector Machines

SVM-Based Cost-sensitive Classification Algorithm with Error Cost and Class-dependent Reject Cost

Binary Classification Algorithm with Class-Dependent Reject Cost

SVM-based Cost Sensitive Mining

Soft Methodology for Cost-and-error Sensitive Classification

Optimizing Cost-Sensitive SVM for Imbalanced Data : Connecting Cluster to Classification.

An empirical study on the performance of cost-sensitive boosting algorithms with different levels of class imbalance

An adaptive Cost-sensitive Classifier

A Statistical Approach to Cost-Sensitive AdaBoost for Imbalanced Data Classification

Cost-Sensitive Alternating Direction Method Of Multipliers For Large-Scale Classification

Using random forest for reliable classification and cost-sensitive learning for medical diagnosis

Hybrid SVM algorithm oriented to classifying imbalanced datasets

Adaptive Cost-Sensitive Online Classification

Abstaining Classification when Error Costs Are Unequal and Unknown.

A study on cost behaviors of binary classification measures in class-imbalanced problems

Cost-Sensitive Classification Using Decision Trees, Boosting and MetaCost