Abstract:Objectives: This study aims to evaluate the fairness performance metrics of Machine Learning (ML) models to predict hospitalization and emergency department (ED) visits in heart failure patients receiving home healthcare. We analyze biases, assess performance disparities, and propose solutions to improve model performance in diverse subpopulations. Methods: The study used a dataset of 12,189 episodes of home healthcare collected between 2015 and 2017, including structured (e.g., standard assessment tool) and unstructured data (i.e., clinical notes). ML risk prediction models, including Light Gradient-boosting model (LightGBM) and AutoGluon, were developed using demographic information, vital signs, comorbidities, service utilization data, and the area deprivation index (ADI) associated with the patient's home address. Fairness metrics, such as Equal Opportunity, Predictive Equality, Predictive Parity, and Statistical Parity, were calculated to evaluate model performance across subpopulations. Results: Our study revealed significant disparities in model performance across diverse demographic subgroups. For example, the Hispanic, Male, High-ADI subgroup excelled in terms of Equal Opportunity with a metric value of 0.825, which was 28% higher than the lowest-performing Other, Female, Low-ADI subgroup, which scored 0.644. In Predictive Parity, the gap between the highest and lowest-performing groups was 29%, and in Statistical Parity, the gap reached 69%. In Predictive Equality, the difference was 45%. Discussion and conclusion: The findings highlight substantial differences in fairness metrics across diverse patient subpopulations in ML risk prediction models for heart failure patients receiving home healthcare services. Ongoing monitoring and improvement of fairness metrics are essential to mitigate biases.

Assessing Fairness in Classification Parity of Machine Learning Models in Healthcare

Algorithmic fairness in artificial intelligence for medicine and healthcare

Algorithm Fairness in AI for Medicine and Healthcare

Fairness in Machine Learning Meets with Equity in Healthcare

What is Fair? Defining Fairness in Machine Learning for Health

A Comparative Study of Fairness in Medical Machine Learning.

An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction

Analyzing Fairness of Classification Machine Learning Model with Structured Dataset

Machine learning and algorithmic fairness in public and population health

Analyzing Fairness of Computer Vision and Natural Language Processing Models

Can AI Help Reduce Disparities in General Medical and Mental Health Care?

Fairness in Classifying and Grouping Health Equity Information

Understanding Fairness Metrics in Recommender Systems: A Healthcare Perspective

Evaluating Fair Feature Selection in Machine Learning for Healthcare

Fairness gaps in Machine learning models for hospitalization and emergency department visit risk prediction in home healthcare patients with heart failure

Fair Bayes-Optimal Classifiers Under Predictive Parity

Connecting Fairness in Machine Learning with Public Health Equity

Fair Machine Learning for Healthcare Requires Recognizing the Intersectionality of Sociodemographic Factors, a Case Study

Equity in Healthcare: Analyzing Disparities in Machine Learning Predictions of Diabetic Patient Readmissions

Fairness in AI for healthcare

A Review on Fairness in Machine Learning