Abstract:Background: Discharge instructions are a key form of documentation and patient communication in the time of transition from the emergency department (ED) to home. Discharge instructions are time-consuming and often underprioritized, especially in the ED, leading to discharge delays and possibly impersonal patient instructions. Generative artificial intelligence and large language models (LLMs) offer promising methods of creating high-quality and personalized discharge instructions; however, there exists a gap in understanding patient perspectives of LLM-generated discharge instructions. Objective: We aimed to assess the use of LLMs such as ChatGPT in synthesizing accurate and patient-accessible discharge instructions in the ED. Methods: We synthesized 5 unique, fictional ED encounters to emulate real ED encounters that included a diverse set of clinician history, physical notes, and nursing notes. These were passed to GPT-4 in Azure OpenAI Service (Microsoft) to generate LLM-generated discharge instructions. Standard discharge instructions were also generated for each of the 5 unique ED encounters. All GPT-generated and standard discharge instructions were then formatted into standardized after-visit summary documents. These after-visit summaries containing either GPT-generated or standard discharge instructions were randomly and blindly administered to Amazon MTurk respondents representing patient populations through Amazon MTurk Survey Distribution. Discharge instructions were assessed based on metrics of interpretability of significance, understandability, and satisfaction. Results: Our findings revealed that survey respondents' perspectives regarding GPT-generated and standard discharge instructions were significantly (P=.01) more favorable toward GPT-generated return precautions, and all other sections were considered noninferior to standard discharge instructions. Of the 156 survey respondents, GPT-generated discharge instructions were assigned favorable ratings, "agree" and "strongly agree," more frequently along the metric of interpretability of significance in discharge instruction subsections regarding diagnosis, procedures, treatment, post-ED medications or any changes to medications, and return precautions. Survey respondents found GPT-generated instructions to be more understandable when rating procedures, treatment, post-ED medications or medication changes, post-ED follow-up, and return precautions. Satisfaction with GPT-generated discharge instruction subsections was the most favorable in procedures, treatment, post-ED medications or medication changes, and return precautions. Wilcoxon rank-sum test of Likert responses revealed significant differences (P=.01) in the interpretability of significant return precautions in GPT-generated discharge instructions compared to standard discharge instructions but not for other evaluation metrics and discharge instruction subsections. Conclusions: This study demonstrates the potential for LLMs such as ChatGPT to act as a method of augmenting current documentation workflows in the ED to reduce the documentation burden of physicians. The ability of LLMs to provide tailored instructions for patients by improving readability and making instructions more applicable to patients could improve upon the methods of communication that currently exist.

PRO-READ IR:Enhanced PROcedural Information READability for Patient-Centered Care in Interventional Radiology with Large Language Models

Using ChatGPT to Improve Readability of Interventional Radiology Procedure Descriptions

Evaluation of ChatGPT-Generated Educational Patient Pamphlets for Common Interventional Radiology Procedures

Evaluation of GPT-4 ability to identify and generate patient instructions for actionable incidental radiology findings

Evaluation of an Artificial Intelligence Chatbot for Delivery of Interventional Radiology Patient Education Material: A Comparison with Societal Website Content.

Large Language Models for Simplified Interventional Radiology Reports: A Comparative Analysis

337 Bridging the Literacy Gap for Surgical Consents: An AI-Human Expert Collaborative Approach

Can generative AI improve the readability of patient education materials at a radiology practice?

Assessing the Readability, Reliability, and Quality of AI-Modified and Generated Patient Education Materials for Endoscopic Skull Base Surgery

ChatGPT for improving postoperative instructions in multiple fields of plastic surgery

Advancing Patient Education in Idiopathic Intracranial Hypertension: The Promise of Large Language Models

Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language Model

Patient-Representing Population's Perceptions of GPT-Generated Versus Standard Emergency Department Discharge Instructions: Randomized Blind Survey Assessment

GPT-4 Improves Readability of Institutional Heart Failure Patient Education Materials

Empowering patients: how accurate and readable are large language models in renal cancer education

Evaluation of Generative Language Models in Personalizing Medical Information: Instrument Validation Study

Improving readability and comprehension levels of otolaryngology patient education materials using ChatGPT

Investigating the capabilities of advanced large language models in generating patient instructions and patient educational material

Assessment of the Quality and Readability of Information Provided by ChatGPT in Relation to the Use of Platelet-Rich Plasma Therapy for Osteoarthritis

The quality and readability of patient information provided by ChatGPT: can AI reliably explain common ENT operations?

Prompt engineering with ChatGPT3.5 and GPT4 to improve patient education on retinal diseases