Abstract:Outcomes 1. Understand how an AI model can augment operational, workflow, and technology redesign to support improvements in serious illness discussions, documentation, and ACP billing. 2. Apply evaluation tools to determine whether an AI model might be appropriate for their clinical context and discuss how an AI mortality model can augment clinician intelligence to improve goal concordant care delivery. Key Message A NCI-designated Cancer Center incorporated an AI model to significantly improve goals of care (GOC) discussions, documentation, and ACP billing with seriously ill patients. The AI model was helpful in identifying almost 60% of priority patients in 2022 and was important in the overall strategy to improve goal concordant care. Importance Advances in therapy within cancer including targeted immunotherapy and CAR-T cell therapies have made it increasingly difficult to predict prognosis. AI risk stratification models may help prioritize the limited capacity of overstretched clinicians to identify patients who would maximally benefit from goals of care discussions. Objectives To provide background on an AI model that outperformed medical oncologists in predicting 90-day mortality for patients with metastatic solid tumors1, provide recommendations on considerations prior to implementing AI within an institution, and show how an NCI-designated Cancer Center incorporated AI within clinical workflows to increase goals of care discussions. Scientific Methods Utilized With broad institutional endorsement, focused multi-disciplinary groups redesigned workflows and the EHR to support automated identification and engagement of advanced cancer patients. Significant EHR customizations were made to automate priority patient identification, including integration of a supervised machine learning 90-day mortality model and evaluation against department-specific clinical criteria. Visual indicators, nudges, and an easy-to-use GOC template with embedded ACP billing were also introduced to improve visibility of priority patients' GOC statuses and ease administrative burden. Results In 2022, 867 priority patients were identified (59% by AI model). Amongst all services, use of ACP billing codes (99497 & 99498) increased 10-fold between 2020 as a baseline year and 2022 (268 to 2650). GOC documentation increased over 7-fold (738 to 5660). Workflows, EHR screenshots, and Tableau dashboards will be shared to demonstrate how AI was incorporated to augment clinicians within workflow. Conclusion(s) AI can be beneficial in supporting a multi-disciplinary effort to implement an Improving Goal Concordant Care Initiative. This led to significant increases in GOC discussions captured within structured documentation in the EHR and through use of ACP billing codes. Impact AI risk stratification can assist goals of care workflows and improve the volume of goals of care discussions to align patient preferences with care. Further work is needed to understand how clinician augmented intelligence can improve performance of risk stratification systems. Keywords Communication and prognostication/Palliative care in oncology

Equity in Using Artificial Intelligence Mortality Predictions to Target Goals of Care Documentation

Using AI to Improve Goal Concordant Care: Deep Dive into a 90-Day Mortality Model and Incorporation into Clinical Care (CO202C)

Evaluation of the Impact of a 90-day Mortality Prediction Model Integrated in the EHR for Advanced Care Planning

The ethical dimensions of utilizing Artificial Intelligence in palliative care

Can an Algorithm Be My Healthcare Proxy?

AI-based clinical decision-making systems in palliative medicine: ethical challenges

Ethics of the algorithmic prediction of goal of care preferences: from theory to practice

Mortality Prediction in Cardiac Intensive Care Unit Patients: A Systematic Review of Existing and Artificial Intelligence Augmented Approaches

Applying Artificial Intelligence to Address the Knowledge Gaps in Cancer Care

The robot butler: How and why should we study predictive algorithms and artificial intelligence (AI) in healthcare?

Artificial Intelligence-Based Models for Prediction of Mortality in ICU Patients: A Scoping Review

Looking Beyond Mortality Prediction: Primary Care Physician Views of Patients’ Palliative Care Needs Predicted by a Machine Learning Tool

An ethical assessment of professional opinions on concerns, chances, and limitations of the implementation of an artificial intelligence-based technology into the geriatric patient treatment and continuity of care

Exploring the literature on artificial intelligence use in oncology.

Using artificial intelligence to promote equitable care for inpatients with language barriers and complex medical needs: clinical stakeholder perspectives

Use of artificial intelligence in critical care: opportunities and obstacles

Ethical use of artificial intelligence to prevent sudden cardiac death: an interview study of patient perspectives

Breathing Life Into Artificial Intelligence*

Artificial intelligence prediction of In-Hospital mortality in patients with dementia: A multi-center study

Machine learning for targeted advance care planning in cancer patients: a quality improvement study

Use of Artificial Intelligence in Improving Outcomes in Heart Disease: A Scientific Statement From the American Heart Association