Abstract:Background: The presence of bias in artificial intelligence has garnered increased attention, with inequities in algorithmic performance being exposed across the fields of criminal justice, education, and welfare services. In health care, the inequitable performance of algorithms across demographic groups may widen health inequalities. Objective: Here, we identify and characterize bias in cardiology algorithms, looking specifically at algorithms used in the management of heart failure. Methods: Stage 1 involved a literature search of PubMed and Web of Science for key terms relating to cardiac machine learning (ML) algorithms. Papers that built ML models to predict cardiac disease were evaluated for their focus on demographic bias in model performance, and open-source data sets were retained for our investigation. Two open-source data sets were identified: (1) the University of California Irvine Heart Failure data set and (2) the University of California Irvine Coronary Artery Disease data set. We reproduced existing algorithms that have been reported for these data sets, tested them for sex biases in algorithm performance, and assessed a range of remediation techniques for their efficacy in reducing inequities. Particular attention was paid to the false negative rate (FNR), due to the clinical significance of underdiagnosis and missed opportunities for treatment. Results: In stage 1, our literature search returned 127 papers, with 60 meeting the criteria for a full review and only 3 papers highlighting sex differences in algorithm performance. In the papers that reported sex, there was a consistent underrepresentation of female patients in the data sets. No papers investigated racial or ethnic differences. In stage 2, we reproduced algorithms reported in the literature, achieving mean accuracies of 84.24% (SD 3.51%) for data set 1 and 85.72% (SD 1.75%) for data set 2 (random forest models). For data set 1, the FNR was significantly higher for female patients in 13 out of 16 experiments, meeting the threshold of statistical significance (-17.81% to -3.37%; P<.05). A smaller disparity in the false positive rate was significant for male patients in 13 out of 16 experiments (-0.48% to +9.77%; P<.05). We observed an overprediction of disease for male patients (higher false positive rate) and an underprediction of disease for female patients (higher FNR). Sex differences in feature importance suggest that feature selection needs to be demographically tailored. Conclusions: Our research exposes a significant gap in cardiac ML research, highlighting that the underperformance of algorithms for female patients has been overlooked in the published literature. Our study quantifies sex disparities in algorithmic performance and explores several sources of bias. We found an underrepresentation of female patients in the data sets used to train algorithms, identified sex biases in model error rates, and demonstrated that a series of remediation techniques were unable to address the inequities present.

Exploring Gender Bias in Cardiovascular Medical Education Through Clinical Simulation

Gender Disparities in Patients’ Decisions about the Management of Myocardial Infarction in East Chinese Province

Racial, Gender, and Socioeconomic Status Bias in Senior Medical Student Clinical Decision-Making: A National Survey

Gender bias in cardiovascular testing persists after adjustment for presenting characteristics and cardiac risk

Overcoming Gender Bias in Assessment of Surgical Skills

Gender Differences in Clinical Performance Assessment of Internal Medicine Residents: A Longitudinal Analysis of the Influence of Faculty and Trainee Gender

A gender biased hidden curriculum of clinical vignettes in undergraduate medical training.

Does gender affect experiences in the operating room for surgeons and anesthesiologists at a large academic center?

'I wouldn't know what to do with the breasts': the impact of patient gender on medical student confidence and comfort in clinical skills

Simulation as an educational tool to teach emergency medicine residents about unconscious bias

Gender Differences in Autonomy and Performance Assessments in a National Cohort of Vascular Surgery Trainees

Association of Gender and Operative Feedback Quality in Surgical Residents

Gender disparities in cardiothoracic surgery: a comparative study in India

Evaluating gender bias in ML-based clinical risk prediction models: A study on multiple use cases at different hospitals

Gender diversity in radiology: Shedding light on the 256 shades of grey

Resident perspectives on the role of gender in operative experience during general surgery residency training: A mixed-methods study

Surgical career choices of medical students in China: does gender bias play a role?

The Gender Divide: The Impact of Surgeon Gender on Surgical Practice Patterns in Urology

Sex-Based Performance Disparities in Machine Learning Algorithms for Cardiac Disease Prediction: Exploratory Study

Clinical Performance Evaluations of Third-Year Medical Students and Association With Student and Evaluator Gender

Competence over confidence: uncovering lower self-efficacy for women residents during central venous catheterization training