NON‐THROMBOGENIC SURFACE INHIBITING PLATELET ADHERENCE

E. Nishizawa,D. J. Wynalda,D. Lednicer

DOI: https://doi.org/10.1097/00002480-197301900-00003

Abstract:

What problem does this paper attempt to address?

Assessing ChatGPT's Proficiency in Simplifying Radiological Reports for Healthcare Professionals and Patients

Pradosh Kumar Sarangi,Amrita Lumbani,M Sarthak Swarup,Suvankar Panda,Smruti Snigdha Sahoo,Pratisruti Hui,Anish Choudhary,Sudipta Mohakud,Ranjan Kumar Patel,Himel Mondal

DOI: https://doi.org/10.7759/cureus.50881

2023-12-21

Cureus

Abstract:Background Clear communication of radiological findings is crucial for effective healthcare decision-making. However, radiological reports are often complex with technical terminology, making them challenging for non-radiology healthcare professionals and patients to comprehend. Large language models like ChatGPT (Chat Generative Pre-trained Transformer, by OpenAI, San Francisco, CA) offer a potential solution by translating intricate reports into simplified language. This study aimed to assess the capability of ChatGPT-3.5 in simplifying radiological reports to facilitate improved understanding by healthcare professionals and patients. Materials and methods Nine radiological reports were taken for this study spanning various imaging modalities and medical conditions. These reports were used to ask ChatGPT a set of seven questions (describe the procedure, mention the key findings, express in a simple language, suggestions for further investigation, need of further investigation, grammatical or typing errors, and translation into Hindi). A total of eight radiologists rated the generated content in detailing, summarizing, simplifying content and language, factual correctness, further investigation, grammatical errors, and translation to Hindi. Results The highest score was obtained for detailing the report (94.17% accuracy) and the lowest score was for drawing conclusions for the patient (85% accuracy); case-wise scores were similar (p-value = 0.97). The Hindi translation by ChatGPT was not suitable for patient communication. Conclusion The current free version of ChatGPT-3.5 was able to simplify radiological reports effectively, removing technical jargon while preserving essential diagnostic information. The free version adeptly simplifies radiological reports, enhancing accessibility for healthcare professionals and patients. Hence, it has the potential to enhance medical communication, facilitating informed decision-making by healthcare professionals and patients.
Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential

Qing Lyu,Josh Tan,Michael E. Zapadka,Janardhana Ponnatapura,Chuang Niu,Kyle J. Myers,Ge Wang,Christopher T. Whitlow

DOI: https://doi.org/10.1186/s42492-023-00136-5

2023-05-18

Abstract:Abstract The large language model called ChatGPT has drawn extensively attention because of its human-like expression and reasoning abilities. In this study, we investigate the feasibility of using ChatGPT in experiments on translating radiology reports into plain language for patients and healthcare providers so that they are educated for improved healthcare. Radiology reports from 62 low-dose chest computed tomography lung cancer screening scans and 76 brain magnetic resonance imaging metastases screening scans were collected in the first half of February for this study. According to the evaluation by radiologists, ChatGPT can successfully translate radiology reports into plain language with an average score of 4.27 in the five-point system with 0.08 places of information missing and 0.07 places of misinformation. In terms of the suggestions provided by ChatGPT, they are generally relevant such as keeping following-up with doctors and closely monitoring any symptoms, and for about 37% of 138 cases in total ChatGPT offers specific suggestions based on findings in the report. ChatGPT also presents some randomness in its responses with occasionally over-simplified or neglected information, which can be mitigated using a more detailed prompt. Furthermore, ChatGPT results are compared with a newly released large model GPT-4, showing that GPT-4 can significantly improve the quality of translated reports. Our results show that it is feasible to utilize large language models in clinical education, and further efforts are needed to address limitations and maximize their potential.

engineering, electrical & electronic,computer science, interdisciplinary applications,imaging science & photographic technology
Translating Radiology Reports into Plain Language using ChatGPT and GPT-4 with Prompt Learning: Promising Results, Limitations, and Potential

Qing Lyu,Josh Tan,Michael E. Zapadka,Janardhana Ponnatapura,Chuang Niu,Kyle J. Myers,Ge Wang,Christopher T. Whitlow

2023-03-29

Abstract:The large language model called ChatGPT has drawn extensively attention because of its human-like expression and reasoning abilities. In this study, we investigate the feasibility of using ChatGPT in experiments on using ChatGPT to translate radiology reports into plain language for patients and healthcare providers so that they are educated for improved healthcare. Radiology reports from 62 low-dose chest CT lung cancer screening scans and 76 brain MRI metastases screening scans were collected in the first half of February for this study. According to the evaluation by radiologists, ChatGPT can successfully translate radiology reports into plain language with an average score of 4.27 in the five-point system with 0.08 places of information missing and 0.07 places of misinformation. In terms of the suggestions provided by ChatGPT, they are general relevant such as keeping following-up with doctors and closely monitoring any symptoms, and for about 37% of 138 cases in total ChatGPT offers specific suggestions based on findings in the report. ChatGPT also presents some randomness in its responses with occasionally over-simplified or neglected information, which can be mitigated using a more detailed prompt. Furthermore, ChatGPT results are compared with a newly released large model GPT-4, showing that GPT-4 can significantly improve the quality of translated reports. Our results show that it is feasible to utilize large language models in clinical education, and further efforts are needed to address limitations and maximize their potential.

Computation and Language,Artificial Intelligence,Medical Physics
Uncovering Language Disparity of ChatGPT in Healthcare: Non-English Clinical Environment for Retinal Vascular Disease Classification (Preprint)

Xiaocong Liu,Jiageng Wu,Anwen Shao,Wei Shen,Panpan Ye,Yao Wang,Juan Ye,Kai Jin,Jie Yang

DOI: https://doi.org/10.2196/preprints.51926

2023-01-01

Abstract:Benefiting from the exceptional ability of text understanding and rich knowledge, large language models (LLMs) like ChatGPT, have shown great potential in English clinical environments. However, the performance of ChatGPT in non-English clinical settings, as well as its reasoning, have not been explored in-depth. To evaluate ChatGPT’s diagnostic performance and inference abilities for retinal vascular diseases in a non-English clinical environment. In this cross-sectional study, we collected 1226 fundus fluorescein angiography reports and corresponding diagnosis written in Chinese, and tested ChatGPT with four prompting strategies (direct diagnosis or diagnosis with explanation and in Chinese or English). ChatGPT using English prompt for direct diagnosis achieved the best performance, with F1-score of 80.05%, which was inferior to ophthalmologists (89.35%) but close to ophthalmologist interns (82.69%). Although ChatGPT can derive reasoning process with a low error rate, mistakes such as misinformation (1.96%), and hallucination (0.59%) still exist. ChatGPT can serve as a helpful medical assistant to provide diagnosis under non-English clinical environments, but there are still performance gaps, language disparity, and errors compared to professionals, which demonstrates the potential limitations and the desiration to continually explore more robust LLMs in ophthalmology practice. ClinicalTrials.gov NCT04718532
The utility of ChatGPT as a generative medical translator

David R Grimm,Yu-Jin Lee,Katherine Hu,Longsha Liu,Omar Garcia,Karthik Balakrishnan,Noel F Ayoub

DOI: https://doi.org/10.1007/s00405-024-08708-8

2024-05-05

Abstract:Purpose: Large language models continue to dramatically change the medical landscape. We aimed to explore the utility of ChatGPT in providing accurate, actionable, and understandable generative medical translations in English, Spanish, and Mandarin pertaining to Otolaryngology. Methods: Responses of GPT-4 to commonly asked patient questions listed on official otolaryngology clinical practice guidelines (CPG) were evaluated with the Patient Education materials Assessment Tool-printable (PEMAT-P.) Additional critical elements were identified a priori to evaluate ChatGPT's accuracy and thoroughness in its responses. Multiple fluent speakers of English, Mandarin, and Spanish evaluated each response generated by ChatGPT. Results: Total PEMAT-P scores differed between English, Mandarin, and Spanish GPT-4 generated responses depicting a moderate effect size of language, Eta-Square 0.07 with scores ranging from 73 to 77 (P-value = 0.03). Overall understandability scores did not differ between English, Mandarin, and Spanish depicting a small effect size of language, Eta-Square 0.02 scores ranging from 76 to 79 (P-value = 0.17), nor did overall actionability scores Eta-Square 0 score ranging 66-73 (P-value = 0.44). Overall a priori procedure-specific responses similarly did not differ between English, Spanish, and Mandarin Eta-Square 0.02 scores ranging 61-78 (P-value = 0.22). Conclusion: GPT-4 produces accurate, understandable, and actionable outputs in English, Spanish, and Mandarin. Responses generated by GPT-4 in Spanish and Mandarin are comparable to English counterparts indicating a novel use for these models within Otolaryngology, and implications for bridging healthcare access and literacy gaps. Level of evidence: IV.
Effectiveness of ChatGPT in explaining complex medical reports to patients

Mengxuan Sun,Ehud Reiter,Anne E Kiltie,George Ramsay,Lisa Duncan,Peter Murchie,Rosalind Adam

2024-06-23

Abstract:Electronic health records contain detailed information about the medical condition of patients, but they are difficult for patients to understand even if they have access to them. We explore whether ChatGPT (GPT 4) can help explain multidisciplinary team (MDT) reports to colorectal and prostate cancer patients. These reports are written in dense medical language and assume clinical knowledge, so they are a good test of the ability of ChatGPT to explain complex medical reports to patients. We asked clinicians and lay people (not patients) to review explanations and responses of ChatGPT. We also ran three focus groups (including cancer patients, caregivers, computer scientists, and clinicians) to discuss output of ChatGPT. Our studies highlighted issues with inaccurate information, inappropriate language, limited personalization, AI distrust, and challenges integrating large language models (LLMs) into clinical workflow. These issues will need to be resolved before LLMs can be used to explain complex personal medical information to patients.

Human-Computer Interaction,Computation and Language,Other Quantitative Biology
Practical Evaluation of ChatGPT Performance for Radiology Report Generation

Mohsen Soleimani,Navisa Seyyedi,Seyed Mohammad Ayyoubzadeh,Sharareh Rostam Niakan Kalhori,Hamidreza Keshavarz

DOI: https://doi.org/10.1016/j.acra.2024.07.020

2024-08-13

Abstract:Rationale and objectives: The process of generating radiology reports is often time-consuming and labor-intensive, prone to incompleteness, heterogeneity, and errors. By employing natural language processing (NLP)-based techniques, this study explores the potential for enhancing the efficiency of radiology report generation through the remarkable capabilities of ChatGPT (Generative Pre-training Transformer), a prominent large language model (LLM). Materials and methods: Using a sample of 1000 records from the Medical Information Mart for Intensive Care (MIMIC) Chest X-ray Database, this investigation employed Claude.ai to extract initial radiological report keywords. ChatGPT then generated radiology reports using a consistent 3-step prompt template outline. Various lexical and sentence similarity techniques were employed to evaluate the correspondence between the AI assistant-generated reports and reference reports authored by medical professionals. Results: Results showed varying performance among NLP models, with Bart (Bidirectional and Auto-Regressive Transformers) and XLM (Cross-lingual Language Model) displaying high proficiency (mean similarity scores up to 99.3%), closely mirroring physician reports. Conversely, DeBERTa (Decoding-enhanced BERT with disentangled attention) and sequence-matching models scored lower, indicating less alignment with medical language. In the Impression section, the Word-Embedding model excelled with a mean similarity of 84.4%, while others like the Jaccard index showed lower performance. Conclusion: Overall, the study highlights significant variations across NLP models in their ability to generate radiology reports consistent with medical professionals' language. Pairwise comparisons and Kruskal-Wallis tests confirmed these differences, emphasizing the need for careful selection and evaluation of NLP models in radiology report generation. This research underscores the potential of ChatGPT to streamline and improve the radiology reporting process, with implications for enhancing efficiency and accuracy in clinical practice.
ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports

Katharina Jeblick,Balthasar Schachtner,Jakob Dexl,Andreas Mittermeier,Anna Theresa Stüber,Johanna Topalis,Tobias Weber,Philipp Wesp,Bastian Oliver Sabel,Jens Ricke,Michael Ingrisch

DOI: https://doi.org/10.1007/s00330-023-10213-1

IF: 7.034

2023-10-06

European Radiology

Abstract:To assess the quality of simplified radiology reports generated with the large language model (LLM) ChatGPT and to discuss challenges and chances of ChatGPT-like LLMs for medical text simplification.

radiology, nuclear medicine & medical imaging
ChatGPT and assistive AI in structured radiology reporting: A systematic review

Ethan Sacoransky,Benjamin Y M Kwan,Donald Soboleski

DOI: https://doi.org/10.1067/j.cpradiol.2024.07.007

2024-07-09

Abstract:Introduction: The rise of transformer-based large language models (LLMs), such as ChatGPT, has captured global attention with recent advancements in artificial intelligence (AI). ChatGPT demonstrates growing potential in structured radiology reporting-a field where AI has traditionally focused on image analysis. Methods: A comprehensive search of MEDLINE and Embase was conducted from inception through May 2024, and primary studies discussing ChatGPT's role in structured radiology reporting were selected based on their content. Results: Of the 268 articles screened, eight were ultimately included in this review. These articles explored various applications of ChatGPT, such as generating structured reports from unstructured reports, extracting data from free text, generating impressions from radiology findings and creating structured reports from imaging data. All studies demonstrated optimism regarding ChatGPT's potential to aid radiologists, though common critiques included data privacy concerns, reliability, medical errors, and lack of medical-specific training. Conclusion: ChatGPT and assistive AI have significant potential to transform radiology reporting, enhancing accuracy and standardization while optimizing healthcare resources. Future developments may involve integrating dynamic few-shot prompting, ChatGPT, and Retrieval Augmented Generation (RAG) into diagnostic workflows. Continued research, development, and ethical oversight are crucial to fully realize AI's potential in radiology.
Uncovering Language Disparity of ChatGPT in Healthcare: Non-English Clinical Environment for Retinal Vascular Disease Classification

Xiaocong Liu,Jiageng Wu,An Shao,Wenyue Shen,Panpan Ye,Yao Wang,Juan Ye,Kai Jin,Jie Yang

DOI: https://doi.org/10.1101/2023.06.28.23291931

2023-01-01

Abstract:Objective To evaluate the effectiveness and reasoning ability of ChatGPT in diagnosing retinal vascular diseases in the Chinese clinical environment. Materials and Methods We collected 1226 fundus fluorescein angiography reports and corresponding diagnosis written in Chinese, and tested ChatGPT with four prompting strategies (direct diagnosis or diagnosis with explanation and in Chinese or English). Results ChatGPT using English prompt for direct diagnosis achieved the best performance, with F1-score of 80.05%, which was inferior to ophthalmologists (89.35%) but close to ophthalmologist interns (82.69%). Although ChatGPT can derive reasoning process with a low error rate, mistakes such as misinformation (1.96%), and hallucination (0.59%) still exist. Discussion and Conclusions ChatGPT can serve as a helpful medical assistant to provide diagnosis under non-English clinical environments, but there are still performance gaps, language disparity, and errors compared to professionals, which demonstrates the potential limitations and the desiration to continually explore more robust LLMs in ophthalmology practice. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement The work is supported by Natural Science Foundation of China (grant number: 82201195). ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Ethics committee/IRB of Second Affiliated Hospital, School of Medicine, Zhejiang University gave ethical approval for this work.(IRB:[NCT04718532][1]) I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Data will be made available for research purposes upon request. Data requests are to be directed to jinkai{at}zju.edu.cn. [1]: /lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT04718532&atom=%2Fmedrxiv%2Fearly%2F2023%2F07%2F14%2F2023.06.28.23291931.atom
Translating musculoskeletal radiology reports into patient-friendly summaries using ChatGPT-4

Ian J. Kuckelman,Karla Wetley,Paul Hyunsoo Yi,Andrew Bailey Ross

DOI: https://doi.org/10.1007/s00256-024-04599-2

2024-01-25

Skeletal Radiology

Abstract:ObjectiveTo assess the feasibility of using large language models (LLMs), specifically ChatGPT-4, to generate concise and accurate layperson summaries of musculoskeletal radiology reports.MethodsSixty radiology reports, comprising 20 MR shoulder, 20 MR knee, and 20 MR lumbar spine reports, were obtained via PACS. The reports were deidentified and then submitted to ChatGPT-4, with the prompt “Produce an organized and concise layperson summary of the findings of the following radiology report. Target a reading level of 8-9th grade and word count <300 words.” Three (two primary and one later added for validation) independent readers evaluated the summaries for completeness and accuracy compared to the original reports. Summaries were rated on a scale of 1 to 3: 1) summaries that were incorrect or incomplete, potentially providing harmful or confusing information; 2) summaries that were mostly correct and complete, unlikely to cause confusion or harm; and 3) summaries that were entirely correct and complete.ResultsAll 60 responses met the criteria for word count and readability. Mean ratings for accuracy were 2.58 for reader 1, 2.71 for reader 2, and 2.77 for reader 3. Mean ratings for completeness were 2.87 for reader 1 and 2.73 for reader 2 and 2.87 for reader 3. For accuracy, reader 1 identified three summaries as a 1, reader 2 identified one, and reader 3 identified none. For the two primary readers, inter-reader agreement was low for accuracy (kappa 0.33) and completeness (kappa 0.29). There were no statistically significant changes in inter-reader agreement when the third reader’s ratings were included in analysis.ConclusionOverall ratings for accuracy and completeness of the AI-generated layperson report summaries were high with only a small minority likely to be confusing or inaccurate. This study illustrates the potential for leveraging generative AI, such as ChatGPT-4, to automate the production of patient-friendly summaries for musculoskeletal MR imaging.

radiology, nuclear medicine & medical imaging,orthopedics
Evaluating the utility of ChatGPT to convert clinic letters into patient friendly language

Simon C Cork,Keith Hopcroft

DOI: https://doi.org/10.1101/2024.07.09.24310132

2024-07-09

Abstract:Communication with patients in language they understand leads to greater comprehension of treatment and diagnoses but can be time consuming for clinicians. Here we sought to investigate the utility of ChatGPT to translate clinic letters into language patients understood, without loss of clinical information. Twenty-three letters from a range of specialities were translated, resulting in no loss of clinical information. Subjective analysis from patient representatives revealed significantly increased understanding of treatment and diagnoses, increased satisfaction, and a significant decrease in the requirement to seek medical assistance in understanding their content when compared to original letters written to clinicians. Overall, we conclude that ChatGPT can be used to translate clinic letters into patient friendly language, and that these letters are preferred by patients.
AI-driven translations for kidney transplant equity in Hispanic populations

Oscar A. Garcia Valencia,Charat Thongprayoon,Caroline C. Jadlowiec,Shennen A. Mao,Napat Leeaphorn,Pooja Budhiraja,Iasmina M. Craici,Maria L. Gonzalez Suarez,Wisit Cheungpasitporn

DOI: https://doi.org/10.1038/s41598-024-59237-7

IF: 4.6

2024-04-14

Scientific Reports

Abstract:Health equity and accessing Spanish kidney transplant information continues being a substantial challenge facing the Hispanic community. This study evaluated ChatGPT's capabilities in translating 54 English kidney transplant frequently asked questions (FAQs) into Spanish using two versions of the AI model, GPT-3.5 and GPT-4.0. The FAQs included 19 from Organ Procurement and Transplantation Network (OPTN), 15 from National Health Service (NHS), and 20 from National Kidney Foundation (NKF). Two native Spanish-speaking nephrologists, both of whom are of Mexican heritage, scored the translations for linguistic accuracy and cultural sensitivity tailored to Hispanics using a 1–5 rubric. The inter-rater reliability of the evaluators, measured by Cohen's Kappa, was 0.85. Overall linguistic accuracy was 4.89 ± 0.31 for GPT-3.5 versus 4.94 ± 0.23 for GPT-4.0 (non-significant p = 0.23). Both versions scored 4.96 ± 0.19 in cultural sensitivity (p = 1.00). By source, GPT-3.5 linguistic accuracy was 4.84 ± 0.37 (OPTN), 4.93 ± 0.26 (NHS), 4.90 ± 0.31 (NKF). GPT-4.0 scored 4.95 ± 0.23 (OPTN), 4.93 ± 0.26 (NHS), 4.95 ± 0.22 (NKF). For cultural sensitivity, GPT-3.5 scored 4.95 ± 0.23 (OPTN), 4.93 ± 0.26 (NHS), 5.00 ± 0.00 (NKF), while GPT-4.0 scored 5.00 ± 0.00 (OPTN), 5.00 ± 0.00 (NHS), 4.90 ± 0.31 (NKF). These high linguistic and cultural sensitivity scores demonstrate Chat GPT effectively translated the English FAQs into Spanish across systems. The findings suggest Chat GPT's potential to promote health equity by improving Spanish access to essential kidney transplant information. Additional research should evaluate its medical translation capabilities across diverse contexts/languages. These English-to-Spanish translations may increase access to vital transplant information for underserved Spanish-speaking Hispanic patients.

multidisciplinary sciences
Bridging the Gap Between Urological Research and Patient Understanding: The Role of Large Language Models in Automated Generation of Layperson's Summaries

Michael B Eppler,Conner Ganjavi,J Everett Knudsen,Ryan J Davis,Oluwatobiloba Ayo-Ajibola,Aditya Desai,Lorenzo Storino Ramacciotti,Andrew Chen,Andre De Castro Abreu,Mihir M Desai,Inderbir S Gill,Giovanni E Cacciamani

DOI: https://doi.org/10.1097/UPJ.0000000000000428

Abstract:Introduction: This study assessed ChatGPT's ability to generate readable, accurate, and clear layperson summaries of urological studies, and compared the performance of ChatGPT-generated summaries with original abstracts and author-written patient summaries to determine its effectiveness as a potential solution for creating accessible medical literature for the public. Methods: Articles from the top 5 ranked urology journals were selected. A ChatGPT prompt was developed following guidelines to maximize readability, accuracy, and clarity, minimizing variability. Readability scores and grade-level indicators were calculated for the ChatGPT summaries, original abstracts, and patient summaries. Two MD physicians independently rated the accuracy and clarity of the ChatGPT-generated layperson summaries. Statistical analyses were conducted to compare readability scores. Cohen's κ coefficient was used to assess interrater reliability for correctness and clarity evaluations. Results: A total of 256 journal articles were included. The ChatGPT-generated summaries were created with an average time of 17.5 (SD 15.0) seconds. The readability scores of the ChatGPT-generated summaries were significantly better than the original abstracts, with Global Readability Score 54.8 (12.3) vs 29.8 (18.5), Flesch Kincade Reading Ease 54.8 (12.3) vs 29.8 (18.5), Flesch Kincaid Grade Level 10.4 (2.2) vs 13.5 (4.0), Gunning Fog Score 12.9 (2.6) vs 16.6 (4.1), Smog Index 9.1 (2.0) vs 12.0 (3.0), Coleman Liau Index 12.9 (2.1) vs 14.9 (3.7), and Automated Readability Index 11.1 (2.5) vs 12.0 (5.7; P < .0001 for all except Automated Readability Index, which was P = .037). The correctness rate of ChatGPT outputs was >85% across all categories assessed, with interrater agreement (Cohen's κ) between 2 independent physician reviewers ranging from 0.76-0.95. Conclusions: ChatGPT can create accurate summaries of scientific abstracts for patients, with well-crafted prompts enhancing user-friendliness. Although the summaries are satisfactory, expert verification is necessary for improved accuracy.
Performance of ChatGPT and Google Translate for Pediatric Discharge Instruction Translation

Ryan C L Brewster,Priscilla Gonzalez,Rohan Khazanchi,Alex Butler,Raquel Selcer,Derrick Chu,Barbara Pontes Aires,Marcella Luercio,Jonathan D Hron

DOI: https://doi.org/10.1542/peds.2023-065573

2024-07-01

PEDIATRICS

Abstract:Background and objectives: Patients who speak languages other than English face barriers to equitable healthcare delivery. Machine translation systems, including emerging large language models, have the potential to expand access to translation services, but their merits and limitations in clinical practice remain poorly defined. We aimed to assess the performance of Google Translate and ChatGPT for multilingual translation of pediatric discharge instructions. Methods: Twenty standardized discharge instructions for pediatric conditions were translated into Spanish, Brazilian Portuguese, and Haitian Creole by professional translation services, Google Translate and ChatGPT-4.0, and evaluated for adequacy (preserved information), fluency (grammatical correctness), meaning (preserved connotation), and severity (clinical harm), along with assessment of overall preference. Domain-level ratings and preferred translation source were summarized with descriptive statistics and compared with professional translations. Results: Google Translate and ChatGPT demonstrated similar domain-level ratings to professional translations for Spanish and Portuguese. For Haitian Creole, compared with both Google Translate and ChatGPT, professional translations demonstrated significantly greater adequacy, fluency meaning, and severity scores. ChatGPT (33.3%, P < .001) and Google Translate (23.3%, P = .024) contained more potentially clinically significant errors (severity score ≤3) for Haitian Creole than professional translations (8.3%). Professional Haitian Creole (48.3%) and Portuguese (43.3%), but not Spanish (15%), translations were most frequently preferred among translation sources. Conclusions: Machine translation platforms have comparable performance to professional translations for Spanish and Portuguese but shortcomings in quality, accuracy, and preference persist for Haitian Creole. Diverse multilingual training data are needed, along with regulations ensuring safe and equitable applications of machine translation in clinical practice.
Artificial Intelligence in Multilingual Interpretation and Radiology Assessment for Clinical Language Evaluation (AI-MIRACLE)

Praneet Khanna,Gagandeep Dhillon,Venkata Buddhavarapu,Ram Verma,Rahul Kashyap,Harpreet Grewal

DOI: https://doi.org/10.3390/jpm14090923

IF: 3.5083

2024-09-02

Journal of Personalized Medicine

Abstract:The AI-MIRACLE Study investigates the efficacy of using ChatGPT 4.0, a large language model (LLM), for translating and simplifying radiology reports into multiple languages, aimed at enhancing patient comprehension. The study assesses the model's performance across the most spoken languages in the U.S., emphasizing the accuracy and clarity of translated and simplified radiology reports for non-medical readers. This study employed ChatGPT 4.0 to translate and simplify selected radiology reports into Vietnamese, Tagalog, Spanish, Mandarin, and Arabic. Hindi was used as a preliminary test language for validation of the questionnaire. Performance was assessed via Google form surveys distributed to bilingual physicians, which assessed the translation accuracy and clarity of simplified texts provided by ChatGPT 4. Responses from 24 participants showed mixed results. The study underscores the model's varying success across different languages, emphasizing both potential applications and limitations. ChatGPT 4.0 shows promise in breaking down language barriers in healthcare settings, enhancing patient comprehension of complex medical information. However, the performance is inconsistent across languages, indicating a need for further refinement and more inclusive training of AI models to handle diverse medical contexts and languages. The study highlights the role of LLMs in improving healthcare communication and patient comprehension, while indicating the need for continued advancements in AI technology, particularly in the translation of low-resource languages.

medicine, general & internal,health care sciences & services
ChatGPT in medicine: A cross-disciplinary systematic review of ChatGPT's (artificial intelligence) role in research, clinical practice, education, and patient interaction

Afia Fatima,Muhammad Ashir Shafique,Khadija Alam,Tagwa Kalool Fadlalla Ahmed,Muhammad Saqlain Mustafa

DOI: https://doi.org/10.1097/MD.0000000000039250

2024-08-09

Abstract:Background: ChatGPT, a powerful AI language model, has gained increasing prominence in medicine, offering potential applications in healthcare, clinical decision support, patient communication, and medical research. This systematic review aims to comprehensively assess the applications of ChatGPT in healthcare education, research, writing, patient communication, and practice while also delineating potential limitations and areas for improvement. Method: Our comprehensive database search retrieved relevant papers from PubMed, Medline and Scopus. After the screening process, 83 studies met the inclusion criteria. This review includes original studies comprising case reports, analytical studies, and editorials with original findings. Result: ChatGPT is useful for scientific research and academic writing, and assists with grammar, clarity, and coherence. This helps non-English speakers and improves accessibility by breaking down linguistic barriers. However, its limitations include probable inaccuracy and ethical issues, such as bias and plagiarism. ChatGPT streamlines workflows and offers diagnostic and educational potential in healthcare but exhibits biases and lacks emotional sensitivity. It is useful in inpatient communication, but requires up-to-date data and faces concerns about the accuracy of information and hallucinatory responses. Conclusion: Given the potential for ChatGPT to transform healthcare education, research, and practice, it is essential to approach its adoption in these areas with caution due to its inherent limitations.
Radiology Gets Chatty: The ChatGPT Saga Unfolds

Harpreet Grewal,Gagandeep Dhillon,Varun Monga,Pranjal Sharma,Venkata S. Buddhavarapu,Gurmanpreet Sidhu,Rahul Kashyap,MBA Gagandeep Dhillon,MBA Rahul Kashyap MBBS,Venkata S Buddhavarapu

DOI: https://doi.org/10.7759/cureus.40135

2023-06-08

Cureus

Abstract:As artificial intelligence (AI) continues to evolve and mature, it is increasingly finding applications in the field of healthcare, particularly in specialties like radiology that are data-heavy and image-focused. Language learning models (LLMs) such as OpenAI's Generative Pre-trained Transformer-4 (GPT-4) are new in the field of medicine and there is a paucity of literature regarding the possible utilities of GPT-4 given its novelty. We aim to present an in-depth exploration of the role of GPT-4, an advanced language model, in radiology. Giving the GPT-4 model prompts for generating reports, template generation, enhancing clinical decision-making, and suggesting captivating titles for research articles, patient communication, and education, can occasionally be quite generic, and at times, it may present factually incorrect content, which could lead to errors. The responses were then analyzed in detail regarding their potential utility in day-to-day radiologist workflow, patient education, and research processes. Further research is required to evaluate LLMs' accuracy and safety in clinical practice and to develop comprehensive guidelines for their implementation.
A thorough examination of ChatGPT-3.5 potential applications in medical writing: A preliminary study

Ismail Dergaa,Helmi Ben Saad,Jordan M. Glenn,Mohamed Ben Aissa,Morteza Taheri,Sarya Swed,Noomen Guelmami,Karim Chamari

DOI: https://doi.org/10.1097/md.0000000000039757

IF: 1.6

2024-10-09

Medicine

Abstract:Effective communication is fundamental to medical research, acting as the primary mechanism for disseminating knowledge, facilitating interdisciplinary collaboration, and driving innovation in healthcare. [ 1 ] In this dynamic and rapidly advancing field, the articulation of complex medical concepts and research outcomes with precision and clarity is crucial. [ 2 ] Recent technological advancements have introduced transformative tools, including large language models such as the Chat Generative Pre-trained Transformer (ChatGPT), which hold significant promise for revolutionizing the domain of medical writing. [ 2–4 ] Many artificial intelligence (AI) chatbots have emerged in the past 3 years, and among them, ChatGPT stands out with its exceptional capabilities in natural language processing and generation. [ 5 ] This technology offers immense potential to revolutionize the way medical professionals communicate their work. [ 5–7 ] Using ChatGPT in medical writing introduces a new way to present complex information and raises important questions about the role of AI in a specialized field like medicine. [ 8–10 ] It promotes a reassessment of conventional writing methods and encourages discussion on how AI may enhance, rather than replace, the knowledge and skills of medical practitioners. Investigating the interaction between human expertise and AI capabilities is crucial for assessing the feasibility and efficacy of ChatGPT in improving medical communication.

medicine, general & internal
Generative Pre-trained Transformer 4 makes cardiovascular magnetic resonance reports easy to understand

Babak Salam,Dmitrij Kravchenko,Sebastian Nowak,Alois M. Sprinkart,Leonie Weinhold,Anna Odenthal,Narine Mesropyan,Leon M. Bischoff,Ulrike Attenberger,Daniel L. Kuetting,Julian A. Luetkens,Alexander Isaak

DOI: https://doi.org/10.1016/j.jocmr.2024.101035

IF: 6.4

2024-01-01

Journal of Cardiovascular Magnetic Resonance

Abstract:BACKGROUND: Patients are increasingly using Generative Pre-trained Transformer 4 (GPT-4) to better understand their own radiology findings.PURPOSE: To evaluate the performance of GPT-4 in transforming cardiovascular magnetic resonance (CMR) reports into text that is comprehensible to medical laypersons.METHODS: ChatGPT with GPT-4 architecture was used to generate three different explained versions of 20 various CMR reports (n = 60) using the same prompt: "Explain the radiology report in a language understandable to a medical layperson". Two cardiovascular radiologists evaluated understandability, factual correctness, completeness of relevant findings, and lack of potential harm, while 13 medical laypersons evaluated the understandability of the original and the GPT-4 reports on a Likert scale (1 "strongly disagree", 5 "strongly agree"). Readability was measured using the Automated Readability Index (ARI). Linear mixed-effects models (values given as median [interquartile range]) and intraclass correlation coefficient (ICC) were used for statistical analysis.RESULTS: GPT-4 reports were generated on average in 52 s ± 13. GPT-4 reports achieved a lower ARI score (10 [9-12] vs 5 [4-6]; p < 0.001) and were subjectively easier to understand for laypersons than original reports (1 [1] vs 4 [4,5]; p < 0.001). Eighteen out of 20 (90%) standard CMR reports and 2/60 (3%) GPT-generated reports had an ARI score corresponding to the 8th grade level or higher. Radiologists' ratings of the GPT-4 reports reached high levels for correctness (5 [4, 5]), completeness (5 [5]), and lack of potential harm (5 [5]); with "strong agreement" for factual correctness in 94% (113/120) and completeness of relevant findings in 81% (97/120) of reports. Test-retest agreement for layperson understandability ratings between the three simplified reports generated from the same original report was substantial (ICC: 0.62; p < 0.001). Interrater agreement between radiologists was almost perfect for lack of potential harm (ICC: 0.93, p < 0.001) and moderate to substantial for completeness (ICC: 0.76, p < 0.001) and factual correctness (ICC: 0.55, p < 0.001).CONCLUSION: GPT-4 can reliably transform complex CMR reports into more understandable, layperson-friendly language while largely maintaining factual correctness and completeness, and can thus help convey patient-relevant radiology information in an easy-to-understand manner.

radiology, nuclear medicine & medical imaging,cardiac & cardiovascular systems

NON‐THROMBOGENIC SURFACE INHIBITING PLATELET ADHERENCE

Assessing ChatGPT's Proficiency in Simplifying Radiological Reports for Healthcare Professionals and Patients

Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential

Translating Radiology Reports into Plain Language using ChatGPT and GPT-4 with Prompt Learning: Promising Results, Limitations, and Potential

Uncovering Language Disparity of ChatGPT in Healthcare: Non-English Clinical Environment for Retinal Vascular Disease Classification (Preprint)

The utility of ChatGPT as a generative medical translator

Effectiveness of ChatGPT in explaining complex medical reports to patients

Practical Evaluation of ChatGPT Performance for Radiology Report Generation

ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports

ChatGPT and assistive AI in structured radiology reporting: A systematic review

Uncovering Language Disparity of ChatGPT in Healthcare: Non-English Clinical Environment for Retinal Vascular Disease Classification

Translating musculoskeletal radiology reports into patient-friendly summaries using ChatGPT-4

Evaluating the utility of ChatGPT to convert clinic letters into patient friendly language

AI-driven translations for kidney transplant equity in Hispanic populations

Bridging the Gap Between Urological Research and Patient Understanding: The Role of Large Language Models in Automated Generation of Layperson's Summaries

Performance of ChatGPT and Google Translate for Pediatric Discharge Instruction Translation

Artificial Intelligence in Multilingual Interpretation and Radiology Assessment for Clinical Language Evaluation (AI-MIRACLE)

ChatGPT in medicine: A cross-disciplinary systematic review of ChatGPT's (artificial intelligence) role in research, clinical practice, education, and patient interaction

Radiology Gets Chatty: The ChatGPT Saga Unfolds

A thorough examination of ChatGPT-3.5 potential applications in medical writing: A preliminary study

Generative Pre-trained Transformer 4 makes cardiovascular magnetic resonance reports easy to understand