The Impact of Race, Ethnicity, and Sex on Fairness in Artificial Intelligence for Glaucoma Prediction Models
Rohith Ravindranath,Joshua D Stein,Tina Hernandez-Boussard,A Caroline Fisher,Sophia Y Wang,SOURCE Consortium,Sejal Amin,Paul A Edwards,Divya Srikumaran,Fasika Woreta,Jeffrey S Schultz,Anurag Shrivastava,Baseer Ahmad,Paul Bryar,Dustin French,Brian L Vanderbeek,Suzann Pershing,Anne M Lynch,Jennifer L Patnaik,Saleha Munir,Wuqaas Munir,Joshua Stein,Lindsey DeLott,Brian C Stagg,Barbara Wirostko,Brian McMillian,Arsham Sheybani,Soshian Sarrapour,Kristen Nwanyanwu,Michael Deiner,Catherine Sun,Houston Robert Feldman,Rajeev Ramachandran
DOI: https://doi.org/10.1016/j.xops.2024.100596
2024-08-14
Abstract:Objective: Despite advances in artificial intelligence (AI) in glaucoma prediction, most works lack multicenter focus and do not consider fairness concerning sex, race, or ethnicity. This study aims to examine the impact of these sensitive attributes on developing fair AI models that predict glaucoma progression to necessitating incisional glaucoma surgery. Design: Database study. Participants: Thirty-nine thousand ninety patients with glaucoma, as identified by International Classification of Disease codes from 7 academic eye centers participating in the Sight OUtcomes Research Collaborative. Methods: We developed XGBoost models using 3 approaches: (1) excluding sensitive attributes as input features, (2) including them explicitly as input features, and (3) training separate models for each group. Model input features included demographic details, diagnosis codes, medications, and clinical information (intraocular pressure, visual acuity, etc.), from electronic health records. The models were trained on patients from 5 sites (N = 27 999) and evaluated on a held-out internal test set (N = 3499) and 2 external test sets consisting of N = 1550 and N = 2542 patients. Main outcomes and measures: Area under the receiver operating characteristic curve (AUROC) and equalized odds on the test set and external sites. Results: Six thousand six hundred eighty-two (17.1%) of 39 090 patients underwent glaucoma surgery with a mean age of 70.1 (standard deviation 14.6) years, 54.5% female, 62.3% White, 22.1% Black, and 4.7% Latinx/Hispanic. We found that not including the sensitive attributes led to better classification performance (AUROC: 0.77-0.82) but worsened fairness when evaluated on the internal test set. However, on external test sites, the opposite was true: including sensitive attributes resulted in better classification performance (AUROC: external #1 - [0.73-0.81], external #2 - [0.67-0.70]), but varying degrees of fairness for sex and race as measured by equalized odds. Conclusions: Artificial intelligence models predicting whether patients with glaucoma progress to surgery demonstrated bias with respect to sex, race, and ethnicity. The effect of sensitive attribute inclusion and exclusion on fairness and performance varied based on internal versus external test sets. Prior to deployment, AI models should be evaluated for fairness on the target population. Financial disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.