Abstract:Shamil E, Ko TK, Fan KS, et al. Assessing the quality and readability of online patient information: ENT UK patient information eleaflets vs responses by a generative artificial intelligence. Facial Plast Surg 2024 (e-pub ahead of print). doi: 10.1055/a-2413-3675. Dear Editor, We would like to comment on the publication by Shamil et al. This study compared the quality and readability of digitally generated health information, specifically ChatGPT, to patient information leaflets created by professionals from the ENT UK using the Ensuring Quality Information for Patients (EQIP) tool for quality assessment and the Flesch–Kincaid Grade Level (FKGL) for readability assessment. The study gathered and examined leaflets from both sources. The results showed that the ENT UK leaflets were of moderate quality, whereas the ChatGPT responses were consistent in quality but less readable. Raters with varying degrees of medical knowledge demonstrated disparities in quality rating, with nonspecialist doctors scoring the highest and medical students scoring the lowest. Overall, the ChatGPT findings revealed comparable content quality, although the readability was worse than those of expert booklets. This study has numerous flaws and limitations. First, the sample size of five ENT UK booklets may be insufficient to draw firm conclusions regarding the overall quality of ChatGPT-generated health information on a wide range of medical topics. A bigger sample size may yield more detailed insights into the trends and consistency in information quality. Furthermore, while the EQIP and FKGL instruments are valuable for assessment, they may not capture all of the information important to patient comprehension. This is especially true for different communities with varying levels of health literacy. This limits our understanding of how the findings apply to real-world patient scenarios. The approach utilized to evaluate participants may generate bias. Raters with various medical knowledge may provide contradictory evaluations to pamphlets, especially when different levels of competence influence perception. This discrepancy is exacerbated by a lack of uniform rating training or anonymization, making it difficult to establish the extent of rater bias in the data. Finally, the mechanism of transmission of health information, such as the format and accessibility of pamphlets versus artificial intelligence (AI) generated responses, was not taken into account, which may have skewed the readability assessments. Future studies should look at a broader range of topics and use larger sample sizes to reach more significant findings about digital health information versus professionally developed health information. Furthermore, using a more refined range of raters, such as a mix of health experts, laypeople, and patients, may aid in better understanding of health information across communities. Examining the effects of various formats on digital health information, such as visual media or multimedia components, may provide more thorough insights for enhancing patient reading and comprehension. This study provides a preliminary methodology for comparing AI-generated health information to professionally established data sources to constructively expand on the findings. Future research could include interactive components in assessments, such as user engagement measurements (e.g., time spent on information and retention rates) and real-world patient comments on usability. Additionally, incorporating machine learning techniques to increase the accessibility and usefulness of AI-generated content could result in better patient education resources tailored to individual requirements. This combination of technology and patient education creates new prospects for improving health care communication in an increasingly digital society. H.D. contributed to 50% ideas, writing, analyzing, and approval. V.W. contributed to 50% ideas, supervision, and approval. The authors used artificial intelligence for language editing of the article. Article published online: 11 October 2024 © 2024. Thieme. All rights reserved. Thieme Medical Publishers, Inc. 333 Seventh Avenue, 18th Floor, New York, NY 10001, USA

Tailored Patient Information: Some Issues and Questions

BT23 Artificial intelligence generation of patient information leaflets can have a useful role in the dermatology clinic

Artificial intelligence-generated patient information leaflets: a comparison of contents according to British Association of Dermatologists standards

One size does not fit all: The case for tailoring print materials

Comment on: Assessing the Quality and Readability of Online Patient Information: ENT UK Patient Information e-Leaflets vs Responses by a Generative Artificial Intelligence

Assessing the Quality and Readability of Online Patient Information: ENT UK Patient Information e-Leaflets versus Responses by a Generative Artificial Intelligence

Evaluation of Generative Language Models in Personalizing Medical Information: Instrument Validation Study

Comparing patient education tools for chronic pain medications: Artificial intelligence chatbot versus traditional patient information leaflets

How to Test Mandatory Text Templates: The European Patient Information Leaflet

Patient information leaflets (PILs) for UK randomised controlled trials: a feasibility study exploring whether they contain information to support decision making about trial participation

Evaluation of an Artificial Intelligence Chatbot for Delivery of Interventional Radiology Patient Education Material: A Comparison with Societal Website Content.

The Challenges for EU User Testing Policies for Patient Information Leaflets

End-of-life Care Patient Information Leaflets-A Comparative Evaluation of Artificial Intelligence-generated Content for Readability, Sentiment, Accuracy, Completeness, and Suitability: ChatGPT vs Google Gemini

PRO-READ IR:Enhanced PROcedural Information READability for Patient-Centered Care in Interventional Radiology with Large Language Models

Acceptability and preliminary feasibility of an internet/CD-ROM-based education and decision program for early-stage prostate cancer patients: randomized pilot study

Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language Model

Utilization of Artificial Intelligence in the Creation of Patient Information on Laryngology Topics

Assessment of the Quality and Readability of Information Provided by ChatGPT in Relation to the Use of Platelet-Rich Plasma Therapy for Osteoarthritis

Can we use ChatGPT for Mental Health and Substance Use Education? Examining Its Quality and Potential Harms

Towards personalized Internet health information: the STEPPS architecture

Suitability of ChatGPT as a Source of Patient Information for Screening Mammography