Abstract:Background: The presence of bias in artificial intelligence has garnered increased attention, with inequities in algorithmic performance being exposed across the fields of criminal justice, education, and welfare services. In health care, the inequitable performance of algorithms across demographic groups may widen health inequalities. Objective: Here, we identify and characterize bias in cardiology algorithms, looking specifically at algorithms used in the management of heart failure. Methods: Stage 1 involved a literature search of PubMed and Web of Science for key terms relating to cardiac machine learning (ML) algorithms. Papers that built ML models to predict cardiac disease were evaluated for their focus on demographic bias in model performance, and open-source data sets were retained for our investigation. Two open-source data sets were identified: (1) the University of California Irvine Heart Failure data set and (2) the University of California Irvine Coronary Artery Disease data set. We reproduced existing algorithms that have been reported for these data sets, tested them for sex biases in algorithm performance, and assessed a range of remediation techniques for their efficacy in reducing inequities. Particular attention was paid to the false negative rate (FNR), due to the clinical significance of underdiagnosis and missed opportunities for treatment. Results: In stage 1, our literature search returned 127 papers, with 60 meeting the criteria for a full review and only 3 papers highlighting sex differences in algorithm performance. In the papers that reported sex, there was a consistent underrepresentation of female patients in the data sets. No papers investigated racial or ethnic differences. In stage 2, we reproduced algorithms reported in the literature, achieving mean accuracies of 84.24% (SD 3.51%) for data set 1 and 85.72% (SD 1.75%) for data set 2 (random forest models). For data set 1, the FNR was significantly higher for female patients in 13 out of 16 experiments, meeting the threshold of statistical significance (-17.81% to -3.37%; P<.05). A smaller disparity in the false positive rate was significant for male patients in 13 out of 16 experiments (-0.48% to +9.77%; P<.05). We observed an overprediction of disease for male patients (higher false positive rate) and an underprediction of disease for female patients (higher FNR). Sex differences in feature importance suggest that feature selection needs to be demographically tailored. Conclusions: Our research exposes a significant gap in cardiac ML research, highlighting that the underperformance of algorithms for female patients has been overlooked in the published literature. Our study quantifies sex disparities in algorithmic performance and explores several sources of bias. We found an underrepresentation of female patients in the data sets used to train algorithms, identified sex biases in model error rates, and demonstrated that a series of remediation techniques were unable to address the inequities present.

Risk of Training Diagnostic Algorithms on Data with Demographic Bias

The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective

Demographic bias in misdiagnosis by computational pathology models

Drop the shortcuts: image augmentation improves fairness and decreases AI detection of race and other demographics from medical images

Algorithmic encoding of protected characteristics in image-based models for disease detection

The Limits of Fair Medical Imaging AI In The Wild

Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis

Achieve Fairness without Demographics for Dermatological Disease Diagnosis

Medical imaging data science competitions should report dataset demographics and evaluate for bias

Discovering mechanisms underlying medical AI prediction of protected attributes

Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data

On Biases in a UK Biobank-based Retinal Image Classification Model

Sex-Based Performance Disparities in Machine Learning Algorithms for Cardiac Disease Prediction: Exploratory Study

Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation

Towards unraveling calibration biases in medical image analysis

Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging

Bias in machine learning models can be significantly mitigated by careful training: Evidence from neuroimaging studies

An AI-Guided Data Centric Strategy to Detect and Mitigate Biases in Healthcare Datasets

Demographic Reporting in Publicly Available Chest Radiograph Data Sets: Opportunities for Mitigating Sex and Racial Disparities in Deep Learning Models

Deep Learning Discovery of Demographic Biomarkers in Echocardiography