Abstract:The selection of an appropriate classification approach depends heavily on the classification rate, which is the most important factor in achieving the desired decision quality. While researchers have examined the impact of different features on the performance of classification approaches, cost/loss functions have received less attention in the comparative literature review, despite their theoretical significance in influencing the classification rate. This paper aims to address this gap by conducting a comparative study on the influence of different cost/loss functions on the classification rate of diverse classifiers. To achieve this objective, the study considers the five most popular and commonly utilized types of cost/loss functions: linear and nonlinear continuous, linear and nonlinear semi-continuous, and discrete cost/loss functions. Furthermore, it takes into consideration the three primary categories of classification approaches: statistical, intelligent, and deep learning classifiers. In addition, a total of 44 benchmark datasets from three distinct domains of medicine, specifically cancer and disease diagnosis, therapy, and biology science, are chosen for analysis. Based on empirical findings, it is evident that the selection of cost/loss functions has a notable impact on the classification rate. The numerical results demonstrate that the discrete cost/loss function performs the best, followed by the semi-continuous and continuous cost/loss functions, in that order. This clearly highlights the positive and direct correlation between aligning the cost/loss function with the goal function of classification approaches and achieving a higher classification rate. Moreover, the average effectiveness of the nonlinear versions of the semi-continuous and continuous cost/loss functions is comparable to that of their linear counterparts. While the choice of cost/loss function can influence the classification rate of various classifiers, the degree of improvement varies depending on the classifier type. In general, statistical classifiers demonstrate a greater degree of enhancement, followed by intelligent classifiers and deep learning models in second and third positions, respectively. Overall, the study reveals a negative correlation between the complexity of classifiers and the improvement in the classification rate when altering the cost/loss function. Furthermore, the numerical findings suggest that the variations in the degree of improvement achieved by changing the cost/loss functions are substantial and affected by the type and domain of the data.

A Comprehensive Survey of Loss Functions in Machine Learning

A survey and taxonomy of loss functions in machine learning

Loss Functions and Metrics in Deep Learning

Searching Parameterized AP Loss for Object Detection

Rank-based Decomposable Losses in Machine Learning: A Survey

On Loss Functions for Deep Neural Networks in Classification

AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks.

Segmentation Loss Odyssey

Generalized robust loss functions for machine learning

Survey of the loss function in classification models: Comparative study in healthcare and medicine

Loss odyssey in medical image segmentation

Loss Function Search for Face Recognition

Loss Function Discovery for Object Detection Via Convergence-Simulation Driven Search

Robustness of different loss functions and their impact on networks learning capability

A Unified View of Loss Functions in Learning to Rank

AutoLoss: Automated Loss Function Search in Recommendations

Tection via convergence-simulation driven search

The Geometry and Calculus of Losses

Auto seg-loss: searching metric surrogates

Alternate Loss Functions for Classification and Robust Regression Can Improve the Accuracy of Artificial Neural Networks