A Novel Artificial Intelligence Platform to Automate Clinical Consultation Notes and Enhance Diagnostic Efficiency in the Outpatient Clinic: Proposal of a Protocol for a Multi-Center, Multi-Disciplinary, Prospective Randomized Controlled Trial

Karanvir Gill,Giovanni Cacciamani,Jamal Nabhani,Joshua Corb,Tom Buchanan,Daniel Park,Virinder Bhardwaj,Onkarjit Marwah,Moses Kim,Deepak Kapoor,Alexander Kutikov,Robert Uzzo,Inderbir Gill

DOI: https://doi.org/10.1101/2023.06.26.23291879

2024-02-29

Abstract:Presented herein is a proposal for a protocol for a multi-center, multi-disciplinary randomized controlled trial (RCT) to evaluate a novel artificial intelligence (AI)-based technology that automates the construction of the clinical consultation note (CCN) and enhances diagnostic assessments in the outpatient clinic setting. This innovative tech-platform automatically generates the CCN and presents it to the provider in advance of the patient consultation, without any work done by the provider. The constructed CCN is presented either in the native electronic health record (EHR) or in a secure web-based application, in a HIPAA-compliant manner. The proposed prospective prospective trial will compare this novel AI/ML technology (NAMT) versus the current standard-of-care (SOC) in the outpatient setting. Outpatient clinic-days will be randomized to either “SOC clinic-day” or the “NAMT clinic-day” based on whether the SOC or the NAMT was used to construct the CCN for all patients seen on that particular clinic-day. Randomized cross-over of each provider between “SOC clinic-day” and “NAMT clinic-day” will result in each provider serving as her/his own internal control. Objective data will be used to compare study endpoints between the SOC and the NAMT. Co-primary endpoints include a) CCN diagnostic accuracy/quality (based on standardized QNOTE metrics); and b) Work-outside-work (WOW) time required by providers to complete clinic-related documentation tasks outside clinic hours (based on EHR meta-data). Secondary endpoints include a) Provider productivity (based on provider “walk-in, walk-out’ time from the consultation room); b) Provider satisfaction (based on the standardized AHRQ EHR End User Survey); and c) Patient satisfaction (based on the standardized Press Ganey/CG-CAHPS survey). To assess generalizability across the health-care spectrum, the study will be conducted in four different types of health-care settings (large academic medical center; non-academic hospital; rural hospital; community private practice); in four different disciplines (cardiology; infectious disease; urology; emergency medicine); using four different EHR systems (Cerner; Epic; AllScripts; MediTech/UroChart). We estimate an aggregate RCT sample size of 150 clinic-days (involving 3,000 total patients; 15-30 providers). This will randomize 75 clinic-days (1,500 patients) to the control SOC arm, and 75 clinic-days (1,500 patients) to the intervention NAMT arm. We will use a two-sided Z-test of difference between proportions with 90% power and two-sided 5% significance level. This RCT is the first to evaluate the efficiency and diagnostic accuracy of pre-constructing CCNs in an automated manner using AI/ML technology, deployed at a large-scale, multi-institutional, multi-disciplinary, multi-EHR level. Results from this study will provide definitive level 1 evidence about the desirability and generalizability of AI-generated automatically constructed CCNs, assessing its potential benefits for providers, patients, and healthcare systems.

Health Systems and Quality Improvement

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to automate the construction of Clinical Consultation Notes (CCN) by introducing a new type of artificial intelligence technology platform, thereby improving the diagnostic efficiency in the outpatient environment and reducing the time required for doctors to complete clinic - related documentation tasks outside of working hours. Specifically, this study aims to evaluate the performance of this new technology platform in the following aspects compared to current standard - of - care practices:

1. **Quality and diagnostic accuracy of CCN**: Use the standardized QNOTE method to evaluate the quality and diagnostic accuracy of CCN.
2. **Work - outside - work (WOW) time**: Measure the time required for doctors to complete documentation tasks outside of normal clinic working hours by analyzing electronic health record (EHR) metadata.

In addition, the study also focuses on the following secondary objectives:

1. **Doctor satisfaction**: Use the standardized AHRQ EHR end - user survey to evaluate doctor satisfaction.
2. **Patient satisfaction**: Use the standardized Press Ganey/CG - CAHPS survey to evaluate patient satisfaction.
3. **Doctor productivity**: Evaluate doctor productivity by recording the time when doctors "walk in and out" of the consultation room and estimate the financial return on investment (ROI).

The study is designed as a multi - center, multi - disciplinary, prospective randomized controlled trial (RCT) and will be carried out in four different types of medical environments, including large academic medical centers, non - academic hospitals, rural hospitals, and community private clinics. The study will evaluate four different specialties (cardiology, infectious diseases, urology, and emergency medicine) and use four different EHR systems (Cerner, Epic, AllScripts, MediTech/UroChart). In this way, the study aims to provide level 1 evidence on the feasibility and universality of AI - generated automatic construction of CCN and evaluate its potential benefits for doctors, patients, and the medical system.