Abstract:Background: Payers and providers still primarily use ordinary least squares (OLS) to estimate expected economic and clinical outcomes for risk adjustment purposes. Penalized linear regression represents a practical and incremental step forward that provides transparency and interpretability within the familiar regression framework. This study conducted an in-depth comparison of prediction performance of standard and penalized linear regression in predicting future health care costs in older adults. Methods and findings: This retrospective cohort study included 81,106 Medicare Advantage patients with 5 years of continuous medical and pharmacy insurance from 2009 to 2013. Total health care costs in 2013 were predicted with comorbidity indicators from 2009 to 2012. Using 2012 predictors only, OLS performed poorly (e.g., R2 = 16.3%) compared to penalized linear regression models (R2 ranging from 16.8 to 16.9%); using 2009-2012 predictors, the gap in prediction performance increased (R2:15.0% versus 18.0-18.2%). OLS with a reduced set of predictors selected by lasso showed improved performance (R2 = 16.6% with 2012 predictors, 17.4% with 2009-2012 predictors) relative to OLS without variable selection but still lagged behind the prediction performance of penalized regression. Lasso regression consistently generated prediction ratios closer to 1 across different levels of predicted risk compared to other models. Conclusions: This study demonstrated the advantages of using transparent and easy-to-interpret penalized linear regression for predicting future health care costs in older adults relative to standard linear regression. Penalized regression showed better performance than OLS in predicting health care costs. Applying penalized regression to longitudinal data increased prediction accuracy. Lasso regression in particular showed superior prediction ratios across low and high levels of predicted risk. Health care insurers, providers and policy makers may benefit from adopting penalized regression such as lasso regression for cost prediction to improve risk adjustment and population health management and thus better address the underlying needs and risk of the populations they serve.

Limitations of P-Values and $R^2$ for Stepwise Regression Building: A Fairness Demonstration in Health Policy Risk Adjustment

Fair regression for health care spending

Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized Odds

Mitigating Included- and Omitted-Variable Bias in Estimates of Disparate Impact

An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction

Counterfactual Reasoning for Fair Clinical Risk Prediction

Social Risk Adjustment In The Hospital Readmissions Reduction Program: A Systematic Review And Implications For Policy

Identifying Undercompensated Groups Defined By Multiple Attributes in Risk Adjustment

Reporting of Fairness Metrics in Clinical Risk Prediction Models: A Call for Change

On (assessing) the fairness of risk score models

Exploring the use of machine learning for risk adjustment: A comparison of standard and penalized linear regression models in predicting health care costs in older adults

High-Dimensional Randomized Crossover Studies: A Clarification of P-Values Interpretation

Enforcing Group Fairness in Algorithmic Decision Making: Utility Maximization Under Sufficiency

Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features

Assessing the Value of Risk Predictions by Using Risk Stratification Tables

An intersectional framework for counterfactual fairness in risk prediction

High-risk pooling for mitigating risk selection incentives in health insurance markets with sophisticated risk equalization: an application based on health survey information

Counterfactual risk assessments, evaluation, and fairness

The Risk Of Perpetuating Health Disparities Through Cost-Effectiveness Analyses

Error Parity Fairness: Testing for Group Fairness in Regression Tasks

Racial Bias in Clinical and Population Health Algorithms: A Critical Review of Current Debates