Qualitative Evaluation of an Artificial Intelligence-based Clinical Decision Support System to Guide Rhythm Management of Atrial Fibrillation (Preprint)

John Stacy,Rachel Kim,Christopher D Barrett,Balaviknesh Sekar,Steve Simon,Farnoush Banaei-Kashani,Michael A. Rosenberg

DOI: https://doi.org/10.2196/preprints.36443

2022-01-14

Abstract:BACKGROUND Despite the numerous studies evaluating various rhythm-control strategies for patients with atrial fibrillation, determination of the optimal strategy in a single patient is often based on trial-and-error, with no one-size-fits-all approach based on international guideline recommendations. The decision, therefore, remains an individualized one that lends itself well to the aid of a clinical decision support system, specifically one guided by artificial intelligence (AI). OBJECTIVE In this study, we performed qualitative evaluation of a novel, AI-based, clinical decision support system (CDSS) for rhythm management of AF called QRhythm, which uses a combination of supervised and reinforcement learning to recommend either a rate-control or one of three types of rhythm-control strategies—external cardioversion, antiarrhythmic medication, or ablation—based on individual patient characteristics. METHODS QRhythm utilizes a two-stage machine learning model to identify the optimal rhythm-management strategy in a given patient based on a set of clinical factors, in which the model first uses supervised learning to mimic an expert clinician, and then applies reinforcement learning to identify the best strategy to obtain the best clinical outcome—a composite of symptomatic recurrence, hospitalization, and stroke. 33 clinical providers, including cardiologists and internal medicine physicians, performed an assessment of QRhythm, followed by a survey to assess relative comfort with automated CDSS in rhythm-management, and examine areas for future development. RESULTS 33 providers were surveyed with training levels ranging from resident to fellow to attending. Of the characteristics of the app surveyed, safety was most important to providers with an average importance rating of 4.7 out of 5 (SD 0.72). This priority was followed by a desire for the advice provided to make clinical sense with an importance of 4.5 out of 5 (SD 0.9), transparency in the population used to create the algorithm, 4.3 out of 5 (SD 0.65), and reasoning behind the decisions made, 4.3 out of 5 (SD0.88), and the ability to challenge the decisions made by the model, 3.85 out of 5 (SD 0.83). Providers that used the app ranked their highest concern with ongoing clinical use of the model as accuracy of recommendations followed by ineffectiveness of the application and patient data security. Trust in the app was varied. 16.7% of providers responded that they somewhat disagreed with the statement “I trust the recommendations provided by the QRhythm app,” 33.3% responded with neutrality to the statement, and 50% somewhat agreed with the statement. CONCLUSIONS Safety of machine learning applications was the number one priority of the providers surveyed, and trust of such models remains varied. Widespread clinical acceptance of machine learning in health care is dependent on how much providers trust the algorithms. Building this trust involves ensuring transparency and interpretability of the model.