Abstract:BACKGROUND: Patients are increasingly using Generative Pre-trained Transformer 4 (GPT-4) to better understand their own radiology findings.PURPOSE: To evaluate the performance of GPT-4 in transforming cardiovascular magnetic resonance (CMR) reports into text that is comprehensible to medical laypersons.METHODS: ChatGPT with GPT-4 architecture was used to generate three different explained versions of 20 various CMR reports (n = 60) using the same prompt: "Explain the radiology report in a language understandable to a medical layperson". Two cardiovascular radiologists evaluated understandability, factual correctness, completeness of relevant findings, and lack of potential harm, while 13 medical laypersons evaluated the understandability of the original and the GPT-4 reports on a Likert scale (1 "strongly disagree", 5 "strongly agree"). Readability was measured using the Automated Readability Index (ARI). Linear mixed-effects models (values given as median [interquartile range]) and intraclass correlation coefficient (ICC) were used for statistical analysis.RESULTS: GPT-4 reports were generated on average in 52 s ± 13. GPT-4 reports achieved a lower ARI score (10 [9-12] vs 5 [4-6]; p < 0.001) and were subjectively easier to understand for laypersons than original reports (1 [1] vs 4 [4,5]; p < 0.001). Eighteen out of 20 (90%) standard CMR reports and 2/60 (3%) GPT-generated reports had an ARI score corresponding to the 8th grade level or higher. Radiologists' ratings of the GPT-4 reports reached high levels for correctness (5 [4, 5]), completeness (5 [5]), and lack of potential harm (5 [5]); with "strong agreement" for factual correctness in 94% (113/120) and completeness of relevant findings in 81% (97/120) of reports. Test-retest agreement for layperson understandability ratings between the three simplified reports generated from the same original report was substantial (ICC: 0.62; p < 0.001). Interrater agreement between radiologists was almost perfect for lack of potential harm (ICC: 0.93, p < 0.001) and moderate to substantial for completeness (ICC: 0.76, p < 0.001) and factual correctness (ICC: 0.55, p < 0.001).CONCLUSION: GPT-4 can reliably transform complex CMR reports into more understandable, layperson-friendly language while largely maintaining factual correctness and completeness, and can thus help convey patient-relevant radiology information in an easy-to-understand manner.

Preliminary assessment of automated radiology report generation with generative pre-trained transformers: comparing results to radiologist-generated reports

Radiology Report Generation Using Transformers Conditioned with Non-imaging Data

Practical Evaluation of ChatGPT Performance for Radiology Report Generation

Generating Radiology Reports via Memory-driven Transformer

Comparative analysis of GPT-4-based ChatGPT's diagnostic performance with radiologists using real-world radiology reports of brain tumors

Generative Pre-trained Transformer 4 makes cardiovascular magnetic resonance reports easy to understand

Generative pre-trained transformer (GPT)-4 support for differential diagnosis in neuroradiology

Preliminary assessment of TNM classification performance for pancreatic cancer in Japanese radiology reports using GPT-4

Comparison of the Diagnostic Performance from Patient's Medical History and Imaging Findings between GPT-4 based ChatGPT and Radiologists in Challenging Neuroradiology Cases

Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers

Evaluating GPT-V4 (GPT-4 with Vision) on Detection of Radiologic Findings on Chest Radiographs

CSAMDT: Conditional Self Attention Memory-Driven Transformers for Radiology Report Generation from Chest X-Ray

MeFormer: Generating Radiology Reports via Memory Enhanced Pretraining Transformer

Automated Radiographic Report Generation Purely on Transformer: A Multicriteria Supervised Approach

Automatic Radiology Report Generator Using Transformer With Contrast-Based Image Enhancement

Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models

Progressive Transformer-Based Generation of Radiology Reports

Clinical Context-aware Radiology Report Generation from Medical Images using Transformers

Multi-modal transformer architecture for medical image analysis and automated report generation

Prior Guided Transformer for Accurate Radiology Reports Generation

Automatic Generation of Chest X-ray Reports Using a Transformer-based Deep Learning Model