Abstract:Regression analysis makes up a large part of supervised machine learning, and consists of the prediction of a continuous independent target from a set of other predictor variables. The difference between binary classification and regression is in the target range: in binary classification, the target can have only two values (usually encoded as 0 and 1), while in regression the target can have multiple values. Even if regression analysis has been employed in a huge number of machine learning studies, no consensus has been reached on a single, unified, standard metric to assess the results of the regression itself. Many studies employ the mean square error (MSE) and its rooted variant (RMSE), or the mean absolute error (MAE) and its percentage variant (MAPE). Although useful, these rates share a common drawback: since their values can range between zero and +infinity, a single value of them does not say much about the performance of the regression with respect to the distribution of the ground truth elements. In this study, we focus on two rates that actually generate a high score only if the majority of the elements of a ground truth group has been correctly predicted: the coefficient of determination (also known as R -squared or R 2 ) and the symmetric mean absolute percentage error (SMAPE). After showing their mathematical properties, we report a comparison between R 2 and SMAPE in several use cases and in two real medical scenarios. Our results demonstrate that the coefficient of determination ( R -squared) is more informative and truthful than SMAPE, and does not have the interpretability limitations of MSE, RMSE, MAE and MAPE. We therefore suggest the usage of R -squared as standard metric to evaluate regression analyses in any scientific domain.

Using the Mean Absolute Percentage Error for Regression Models

Mean Absolute Percentage Error for regression models

Empirical risk minimization is consistent with the mean absolute percentage error

A better measure of relative prediction accuracy for model selection and model estimation

Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not

On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression

The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation

A simulation study of the model evaluation criterion mmre

Least Absolute Relative Error Estimation

Error Density-dependent Empirical Risk Minimization

A Relative Error Estimation Approach for Multiplicative Single Index Model

Nonconcave Penalized M-estimation for the Least Absolute Relative Errors Model

Mean of Ratios or Ratio of Means: statistical uncertainty applied to estimate Multiperiod Probability of Defaul

Model aggregation: minimizing empirical variance outperforms minimizing empirical error

Insights into the Robustness of Minimum Error Entropy Estimation

Comparisons of two ensemble mean methods in measuring the average error growth and the predictability

Optimal estimator for uncertainty-based measurement quality control

A New Interpretation on the MMSE As a Robust MEE Criterion

Least Product Relative Error Estimation

Fast Minimum Error Entropy for Linear Regression

Empirical Risk Minimization with Relative Entropy Regularization