Adjust Pearson's r to Measure Nonlinear Monotone Dependence

Xinbo Ai

DOI: https://doi.org/10.48550/arXiv.2205.04571

2024-04-01

Abstract:As the most commonly used correlation coefficient, Pearson's r is generally considered as a measure only for linear dependence. The use of Pearson's r has been strongly discouraged for nonlinear relationships. Recent research shows that this opinion is myth and Pearson's r should not be ruled out a priori for measuring nonlinear monotone relationships. We find that Pearson's r does have potential to measure nonlinear dependence as long as the relationships are monotone, it just underestimates them. We derive a new inequality which is tighter than the famous Cauchy-Schwarz Inequality, leverage it to adjust the underestimation, and propose a new correlation coefficient, i.e., rearrangement correlation. The rearrangement correlation is able to capture arbitrary monotone relationships, both linear and nonlinear ones. It reverts to Pearson's r in linear scenarios. Simulation experiments and real-life investigations show that the rearrangement correlation is more accurate in measuring nonlinear monotone dependence than the three classical correlation coefficients, and other recently proposed dependence measures.

Other Statistics