Abstract:Shamil E, Ko TK, Fan KS, et al. Assessing the quality and readability of online patient information: ENT UK patient information eleaflets vs responses by a generative artificial intelligence. Facial Plast Surg 2024 (e-pub ahead of print). doi: 10.1055/a-2413-3675. Dear Editor, We would like to comment on the publication by Shamil et al. This study compared the quality and readability of digitally generated health information, specifically ChatGPT, to patient information leaflets created by professionals from the ENT UK using the Ensuring Quality Information for Patients (EQIP) tool for quality assessment and the Flesch–Kincaid Grade Level (FKGL) for readability assessment. The study gathered and examined leaflets from both sources. The results showed that the ENT UK leaflets were of moderate quality, whereas the ChatGPT responses were consistent in quality but less readable. Raters with varying degrees of medical knowledge demonstrated disparities in quality rating, with nonspecialist doctors scoring the highest and medical students scoring the lowest. Overall, the ChatGPT findings revealed comparable content quality, although the readability was worse than those of expert booklets. This study has numerous flaws and limitations. First, the sample size of five ENT UK booklets may be insufficient to draw firm conclusions regarding the overall quality of ChatGPT-generated health information on a wide range of medical topics. A bigger sample size may yield more detailed insights into the trends and consistency in information quality. Furthermore, while the EQIP and FKGL instruments are valuable for assessment, they may not capture all of the information important to patient comprehension. This is especially true for different communities with varying levels of health literacy. This limits our understanding of how the findings apply to real-world patient scenarios. The approach utilized to evaluate participants may generate bias. Raters with various medical knowledge may provide contradictory evaluations to pamphlets, especially when different levels of competence influence perception. This discrepancy is exacerbated by a lack of uniform rating training or anonymization, making it difficult to establish the extent of rater bias in the data. Finally, the mechanism of transmission of health information, such as the format and accessibility of pamphlets versus artificial intelligence (AI) generated responses, was not taken into account, which may have skewed the readability assessments. Future studies should look at a broader range of topics and use larger sample sizes to reach more significant findings about digital health information versus professionally developed health information. Furthermore, using a more refined range of raters, such as a mix of health experts, laypeople, and patients, may aid in better understanding of health information across communities. Examining the effects of various formats on digital health information, such as visual media or multimedia components, may provide more thorough insights for enhancing patient reading and comprehension. This study provides a preliminary methodology for comparing AI-generated health information to professionally established data sources to constructively expand on the findings. Future research could include interactive components in assessments, such as user engagement measurements (e.g., time spent on information and retention rates) and real-world patient comments on usability. Additionally, incorporating machine learning techniques to increase the accessibility and usefulness of AI-generated content could result in better patient education resources tailored to individual requirements. This combination of technology and patient education creates new prospects for improving health care communication in an increasingly digital society. H.D. contributed to 50% ideas, writing, analyzing, and approval. V.W. contributed to 50% ideas, supervision, and approval. The authors used artificial intelligence for language editing of the article. Article published online: 11 October 2024 © 2024. Thieme. All rights reserved. Thieme Medical Publishers, Inc. 333 Seventh Avenue, 18th Floor, New York, NY 10001, USA

Improving readability and comprehension levels of otolaryngology patient education materials using ChatGPT

The Use of Artificial Intelligence to Improve Readability of Otolaryngology Patient Education Materials

Utilizing Artificial Intelligence to Increase the Readability of Patient Education Materials in Pediatric Otolaryngology

Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language Model

Utilization of Artificial Intelligence in the Creation of Patient Information on Laryngology Topics

Assessing the Readability, Reliability, and Quality of AI-Modified and Generated Patient Education Materials for Endoscopic Skull Base Surgery

The Use of Large Language Models to Generate Education Materials about Uveitis

Expanding Accessibility in Cleft Care: The Role of Artificial Intelligence in Improving Literacy of Alveolar Bone Grafting Information

Can generative AI improve the readability of patient education materials at a radiology practice?

Evaluation of Generative Language Models in Personalizing Medical Information: Instrument Validation Study

Bridging Gaps in Health Literacy for Cleft Lip and Palate: The Role of Artificial Intelligence and Interactive Educational Materials

Enhancing Readability of Online Patient-Facing Content: The Role of AI Chatbots in Improving Cancer Information Accessibility

New Frontiers in Health Literacy: Using ChatGPT to Simplify Health Information for People in the Community

Using Large Language Models to Generate Educational Materials on Childhood Glaucoma

Comparing the Ability of Google and ChatGPT to Accurately Respond to Oculoplastics-Related Patient Questions and Generate Customized Oculoplastics Patient Education Materials

Assessing the Quality and Readability of Online Patient Information: ENT UK Patient Information e-Leaflets versus Responses by a Generative Artificial Intelligence

Assessing the Readability of Patient Education Materials on Cardiac Catheterization From Artificial Intelligence Chatbots: An Observational Cross-Sectional Study

Utility of ChatGPT for Automated Creation of Patient Education Handouts: An Application in Neuro-Ophthalmology

Can ChatGPT help patients answer their otolaryngology questions?

Comment on: Assessing the Quality and Readability of Online Patient Information: ENT UK Patient Information e-Leaflets vs Responses by a Generative Artificial Intelligence

The quality and readability of patient information provided by ChatGPT: can AI reliably explain common ENT operations?