The AI Agent in the Room: Informing Objective Decision Making at the Transplant Selection Committee

Bima J Hasjim,Ghazal Azafar,Frank G Lee,Tayyab S Diwan,Shilpa Raju,Jed Adam Gross,Aman Sidhu,Hirohito Ichii,Rahul G Krishnan,Muhammad Mamdani,Divya Sharma,Mamatha Bhat

DOI: https://doi.org/10.1101/2024.12.06.24318575

2024-12-08

Abstract:Importance: Transplantation is one of the few areas in medicine where the definitive treatment is rationed. Subjective decision-making pose challenges towards the transplant selection process. It has been proposed that large language models (LLMs) as artificial intelligent (AI) agents could provide objectivity in decision-making to solve complex problems. Objective: To examine the performance of a multidisciplinary selection committee of AI agents (AI-SC) as a proof-of-concept towards objectivity in the liver transplant (LT) selection process. Design: The AI-SC consisted of four LLMs: transplant hepatologist, transplant surgeon, cardiologist, and social worker. Zero-shot prompting with chain-of thought was used. Decisions were made based on clinicodemographic characteristics at time of waitlisting and LT. Setting: National LT cohort. Participants: Adult patients receiving deceased donor LT from 2004-2023 were extracted from the Scientific Registry of Transplant Recipients (SRTR) and clinical vignettes were generated. Standard absolute contraindications to LT were randomly assigned to a subset of patients to expose the AI-SC to cases of patients declined for LT. Exposures: Clinicodemographic characteristics at waitlisting and transplantation. Main Outcomes and Measures: The AI-SCs accuracy with either: 1) listing candidates if LT would offer a 6-month or 1-year survival benefit or 2) declining candidates if contraindications to LT are present or if LT would not offer those survival benefits. Results: Of 8,412 patients, 83.6% were waitlisted and 16.4% had contraindications to LT. The AI-SC was able to accurately identify contraindications to LT (accuracy: 98.2%, 95%CI 97.9%-98.4%), predict 6-month (94.9%, 95%CI 94.4%-95.3%) and 1-year (92.0%, 95%CI 91.4%-92.6%) survival. HCC burden beyond Milan criteria was the most common reason for accepted patients who were declined by AI-SC (False Negative). Malignancy was the most common cause of death prior to 6-month or 1-year end points (False Positive). The AI-SC most frequently did not perceive a lack of social support or severe cardiopulmonary disease as barriers to LT. Conclusions and Relevance: LLMs can be leveraged to simulate the LT-SC meetings and provide accurate, objective insights on patients who may or may not benefit from LT. Lessons learned from this proof-of-concept are a provocative step towards making the LT selection process more equitable and objective.