A strategy to improve the health of children and young people.

A. Glasper

DOI: https://doi.org/10.12968/BJON.2012.21.17.1040

British Journal of Nursing

Abstract:Professor Alan Glasper discusses how proposals by the Children and Young People's Health Outcomes Forum seek to improve the health of children and young people.

What problem does this paper attempt to address?

A Large Language Model Screening Tool to Target Patients for Best Practice Alerts: Development and Validation

Thomas Savage,John Wang,Lisa Shieh

DOI: https://doi.org/10.2196/49886

IF: 3.2

2023-11-28

JMIR Medical Informatics

Abstract:Background: Best Practice Alerts (BPAs) are alert messages to physicians in the electronic health record that are used to encourage appropriate use of health care resources. While these alerts are helpful in both improving care and reducing costs, BPAs are often broadly applied nonselectively across entire patient populations. The development of large language models (LLMs) provides an opportunity to selectively identify patients for BPAs. Objective: In this paper, we present an example case where an LLM screening tool is used to select patients appropriate for a BPA encouraging the prescription of deep vein thrombosis (DVT) anticoagulation prophylaxis. The artificial intelligence (AI) screening tool was developed to identify patients experiencing acute bleeding and exclude them from receiving a DVT prophylaxis BPA. Methods: Our AI screening tool used a BioMed-RoBERTa (Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach; AllenAI) model to perform classification of physician notes, identifying patients without active bleeding and thus appropriate for a thromboembolism prophylaxis BPA. The BioMed-RoBERTa model was fine-tuned using 500 history and physical notes of patients from the MIMIC-III (Medical Information Mart for Intensive Care) database who were not prescribed anticoagulation. A development set of 300 MIMIC patient notes was used to determine the model's hyperparameters, and a separate test set of 300 patient notes was used to evaluate the screening tool. Results: Our MIMIC-III test set population of 300 patients included 72 patients with bleeding (ie, were not appropriate for a DVT prophylaxis BPA) and 228 without bleeding who were appropriate for a DVT prophylaxis BPA. The AI screening tool achieved impressive accuracy with a precision-recall area under the curve of 0.82 (95% CI 0.75-0.89) and a receiver operator curve area under the curve of 0.89 (95% CI 0.84-0.94). The screening tool reduced the number of patients who would trigger an alert by 20% (240 instead of 300 alerts) and increased alert applicability by 14.8% (218 [90.8%] positive alerts from 240 total alerts instead of 228 [76%] positive alerts from 300 total alerts), compared to nonselectively sending alerts for all patients. Conclusions: These results show a proof of concept on how language models can be used as a screening tool for BPAs. We provide an example AI screening tool that uses a HIPAA (Health Insurance Portability and Accountability Act)–compliant BioMed-RoBERTa model deployed with minimal computing power. Larger models (eg, Generative Pre-trained Transformers–3, Generative Pre-trained Transformers–4, and Pathways Language Model) will exhibit superior performance but require data use agreements to be HIPAA compliant. We anticipate LLMs to revolutionize quality improvement in hospital medicine.

medical informatics
Large language models for extracting histopathologic diagnoses from electronic health records

Brian Johnson,Tyler Bath,Xinyi Huang,Mark Lamm,Ashley Earles,Hyrum Eddington,Lily J. Jih,Samir Gupta,Shailja C. Shah,Kit Curtius

DOI: https://doi.org/10.1101/2024.11.27.24318083

2024-11-28

Abstract:Background & Aims Accurate data resources are essential for impactful medical research. To date, most large-scale studies have relied on structured sources, such as International Classification of Diseases codes, to determine patient diagnoses and outcomes. However, these structured datasets are often incomplete or inaccurate. Recent advances in natural language processing, specifically the introduction of open-weight large language models (LLMs), enable more accurate data extraction from unstructured text in electronic health records (EHRs). Methods We created an approach using LLMs for identifying histopathologic diagnoses, including presence of dysplasia and cancer, in pathology reports from the Department of Veterans Affairs Healthcare System, including those patients with genotype data within the Million Veteran Program (MVP) biobank. Our approach requires no additional training and utilizes a simple 'yes/no' question prompt to obtain an answer. We validated the method on 3 diagnostic tasks by applying the same prompts to reports from patients with vs without diagnoses of inflammatory bowel disease (IBD) and calculating F-1 scores as a balanced accuracy measure. Results In patients without IBD in MVP, we achieved F1-scores of 99.3% for identifying any dysplasia, 98.2% for identifying high-grade dysplasia and/or colorectal adenocarcinoma (HGD/CRC), and 96.2% for identifying CRC using LLM Gemma-2. In IBD patients in MVP, we achieved F1-scores of 97.1% for identifying dysplasia, 96.4% for identifying HGD/CRC, and 97.1% for identifying CRC. Conclusion LLMs provide excellent accuracy in extracting diagnoses from EHRs and can be applied to a variety of tasks with no additional human-led development required. Our validated methods generalized to unstructured pathology notes, even withstanding challenges of resource-limited computing environments.

Health Informatics
Early identification of patients with acute gastrointestinal bleeding using natural language processing and decision rules

Dennis Shung,Cynthia Tsay,Loren Laine,David Chang,Fan Li,Prem Thomas,Caitlin Partridge,Michael Simonov,Allen Hsiao,J Kenneth Tay,Andrew Taylor

DOI: https://doi.org/10.1111/jgh.15313

Abstract:Background and aim: Guidelines recommend risk stratification scores in patients presenting with gastrointestinal bleeding (GIB), but such scores are uncommonly employed in practice. Automation and deployment of risk stratification scores in real time within electronic health records (EHRs) would overcome a major impediment. This requires an automated mechanism to accurately identify ("phenotype") patients with GIB at the time of presentation. The goal is to identify patients with acute GIB by developing and evaluating EHR-based phenotyping algorithms for emergency department (ED) patients. Methods: We specified criteria using structured data elements to create rules for identifying patients and also developed multiple natural language processing (NLP)-based approaches for automated phenotyping of patients, tested them with tenfold cross-validation for 10 iterations (n = 7144) and external validation (n = 2988) and compared them with a standard method to identify patient conditions, the Systematized Nomenclature of Medicine. The gold standard for GIB diagnosis was the independent dual manual review of medical records. The primary outcome was the positive predictive value. Results: A decision rule using GIB-specific terms from ED triage and ED review-of-systems assessment performed better than the Systematized Nomenclature of Medicine on internal validation and external validation (positive predictive value = 85% confidence interval:83%-87% vs 69% confidence interval:66%-72%; P < 0.001). The syntax-based NLP algorithm and Bidirectional Encoder Representation from Transformers neural network-based NLP algorithm had similar performance to the structured-data fields decision rule. Conclusions: An automated decision rule employing GIB-specific triage and review-of-systems terms can be used to trigger EHR-based deployment of risk stratification models to guide clinical decision making in real time for patients with acute GIB presenting to the ED.
Large language models for accurate disease detection in electronic health records

Nils Burgisser,Etienne Chalot,Samia Mehouachi,Clement P. Buclin,Kim Lauper,Delphine Sophie Courvoisier,Denis Mongin

DOI: https://doi.org/10.1101/2024.07.27.24311106

2024-07-29

Abstract:Importance: The use of large language models (LLMs) in medicine is increasing, with potential applications in electronic health records (EHR) to create patient cohorts or identify patients who meet clinical trial recruitment criteria. However, significant barriers remain, including the extensive computer resources required, lack of performance evaluation, and challenges in implementation. Objective: This study aims to propose and test a framework to detect disease diagnosis using a recent light LLM on French-language EHR documents. Specifically, it focuses on detecting gout ( in French), a ubiquitous French term that have multiple meanings beyond the disease. The study will compare the performance of the LLM-based framework with traditional natural language processing techniques and test its dependence on the parameter used. Design: The framework was developed using a training and testing set of 700 paragraphs assessing , issued from a random selection of retrospective EHR documents. All paragraphs were manually reviewed and classified by two health-care professionals (HCP) into disease (true gout) and non-disease (gold standard). The LLM's accuracy was tested using few-shot and chain-of-thought prompting and compared to a regular expression (regex)-based method, focusing on the effects of model parameters and prompt structure. The framework was further validated on 600 paragraphs assessing . Setting: The documents were sampled from the electronic health-records of a tertiary university hospital in Geneva, Switzerland. Participants: Adults over 18 years of age. Exposure: Meta's Llama 3 8B LLM or traditional method, against a gold standard. Main Outcomes and Measures: Positive and negative predictive value, as well as accuracy of tested models. Results: The LLM-based algorithm outperformed the regex method, achieving a 92.7% [88.7-95.4%] positive predictive value, a 96.6% [94.6-97.8%] negative predictive value, and an accuracy of 95.4% [93.6-96.7%] for gout. In the validation set on CPPD, accuracy was 94.1% [90.2-97.6%]. The LLM framework performed well over a wide range of parameter values. Conclusions and Relevance: LLMs were able to accurately detect disease diagnoses from EHRs, even in non-English languages. They could facilitate creating large disease registries in any language, improving disease care assessment and patient recruitment for clinical trials.

Rheumatology
Artificial Intelligence in Gastrointestinal Bleeding Analysis for Video Capsule Endoscopy: Insights, Innovations, and Prospects (2008-2023)

Tanisha Singh,Shreshtha Jha,Nidhi Bhatt,Palak Handa,Nidhi Goel,Sreedevi Indu

2024-09-01

Abstract:The escalating global mortality and morbidity rates associated with gastrointestinal (GI) bleeding, compounded by the complexities and limitations of traditional endoscopic methods, underscore the urgent need for a critical review of current methodologies used for addressing this condition. With an estimated 300,000 annual deaths worldwide, the demand for innovative diagnostic and therapeutic strategies is paramount. The introduction of Video Capsule Endoscopy (VCE) has marked a significant advancement, offering a comprehensive, non-invasive visualization of the digestive tract that is pivotal for detecting bleeding sources unattainable by traditional methods. Despite its benefits, the efficacy of VCE is hindered by diagnostic challenges, including time-consuming analysis and susceptibility to human error. This backdrop sets the stage for exploring Machine Learning (ML) applications in automating GI bleeding detection within capsule endoscopy, aiming to enhance diagnostic accuracy, reduce manual labor, and improve patient outcomes. Through an exhaustive analysis of 113 papers published between 2008 and 2023, this review assesses the current state of ML methodologies in bleeding detection, highlighting their effectiveness, challenges, and prospective directions. It contributes an in-depth examination of AI techniques in VCE frame analysis, offering insights into open-source datasets, mathematical performance metrics, and technique categorization. The paper sets a foundation for future research to overcome existing challenges, advancing gastrointestinal diagnostics through interdisciplinary collaboration and innovation in ML applications.

Computer Vision and Pattern Recognition,Artificial Intelligence
Optimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes

Mauro Giuffrè,Simone Kresevic,Nicola Pugliese,Kisung You,Dennis L. Shung

DOI: https://doi.org/10.1111/liv.15974

IF: 8.754

2024-06-01

Liver International

Abstract:Large Language Models (LLMs) are transformer‐based neural networks with billions of parameters trained on very large text corpora from diverse sources. LLMs have the potential to improve healthcare due to their capability to parse complex concepts and generate context‐based responses. The interest in LLMs has not spared digestive disease academics, who have mainly investigated foundational LLM accuracy, which ranges from 25% to 90% and is influenced by the lack of standardized rules to report methodologies and results for LLM‐oriented research. In addition, a critical issue is the absence of a universally accepted definition of accuracy, varying from binary to scalar interpretations, often tied to grader expertise without reference to clinical guidelines. We address strategies and challenges to increase accuracy. In particular, LLMs can be infused with domain knowledge using Retrieval Augmented Generation (RAG) or Supervised Fine‐Tuning (SFT) with reinforcement learning from human feedback (RLHF). RAG faces challenges with in‐context window limits and accurate information retrieval from the provided context. SFT, a deeper adaptation method, is computationally demanding and requires specialized knowledge. LLMs may increase patient quality of care across the field of digestive diseases, where physicians are often engaged in screening, treatment and surveillance for a broad range of pathologies for which in‐context learning or SFT with RLHF could improve clinical decision‐making and patient outcomes. However, despite their potential, the safe deployment of LLMs in healthcare still needs to overcome hurdles in accuracy, suggesting a need for strategies that integrate human feedback with advanced model training.

gastroenterology & hepatology
Critical Care Studies Using Large Language Models Based on Electronic Healthcare Records: A Technical Note

Zhongheng Zhang,Hongying Ni

DOI: https://doi.org/10.1016/j.jointm.2024.09.002

2024-01-01

Journal of Intensive Medicine

Abstract:The integration of large language models (LLMs) in clinical medicine, particularly in critical care, has introduced transformative capabilities for analyzing and managing complex medical information. This technical note explores the application of LLMs, such as generative pretrained transformer 4 (GPT-4) and Qwen-Chat, in interpreting electronic healthcare records to assist with rapid patient condition assessments, predict sepsis, and automate the generation of discharge summaries. The note emphasizes the significance of LLMs in processing unstructured data from electronic health records (EHRs), extracting meaningful insights, and supporting personalized medicine through nuanced understanding of patient histories. Despite the technical complexity of deploying LLMs in clinical settings, this document provides a comprehensive guide to facilitate the effective integration of LLMs into clinical workflows, focusing on the use of DashScope's application programming interface (API) services for judgment on patient prognosis and organ support recommendations based on natural language in EHRs. By illustrating practical steps and best practices, this work aims to lower the technical barriers for clinicians and researchers, enabling broader adoption of LLMs in clinical research and practice to enhance patient care and outcomes.
Automated Multimodal Machine Learning for Esophageal Variceal Bleeding Prediction Based on Endoscopy and Structured Data

Yu Wang,Yu Hong,Yue Wang,Xin Zhou,Xin Gao,Chenyan Yu,Jiaxi Lin,Lu Liu,Jingwen Gao,Minyue Yin,Guoting Xu,Xiaolin Liu,Jinzhou Zhu

DOI: https://doi.org/10.1007/s10278-022-00724-6

IF: 4.903

2022-10-25

Journal of Digital Imaging

Abstract:Esophageal variceal (EV) bleeding is a severe medical emergency related to cirrhosis. Early identification of cirrhotic patients with at a high risk of EV bleeding is key to improving outcomes and optimizing medical resources. This study aimed to evaluate the feasibility of automated multimodal machine learning (MMML) for predicting EV bleeding by integrating endoscopic images and clinical structured data. This study mainly includes three steps: step 1, developing deep learning (DL) models using EV images by 12-month bleeding on TensorFlow (backbones include ResNet, Xception, EfficientNet, ViT and ConvMixer); step 2, training and internally validating MMML models integrating clinical structured data and DL model outputs to predict 12-month EV bleeding on an H2O-automated machine learning platform (algorithms include DL, XGBoost, GLM, GBM, RF, and stacking); and step 3, externally testing MMML models. Furthermore, existing clinical indices, e.g., the MELD score, Child‒Pugh score, APRI, and FIB-4, were also examined. Five DL models were transfer learning to the binary classification of EV endoscopic images at admission based on the occurrence or absence of bleeding events during the 12-month follow-up. An EfficientNet model achieved the highest accuracy of 0.868 in the validation set. Then, a series of MMML models, integrating clinical structured data and the output of the EfficientNet model, were automatedly trained to predict 12-month EV bleeding. A stacking model showed the highest accuracy (0.932), sensitivity (0.952), and F1-score (0.879) in the test dataset, which was also better than the existing indices. This study is the first to evaluate the feasibility of automated MMML in predicting 12-month EV bleeding based on endoscopic images and clinical variables.

radiology, nuclear medicine & medical imaging
Algorithmic Identification of Treatment‐Emergent Adverse Events From Clinical Notes Using Large Language Models: A Pilot Study in Inflammatory Bowel Disease

Anna L. Silverman,Madhumita Sushil,Balu Bhasuran,Dana Ludwig,James Buchanan,Rebecca Racz,Mahalakshmi Parakala,Samer El‐Kamary,Ohenewaa Ahima,Artur Belov,Lauren Choi,Monisha Billings,Yan Li,Nadia Habal,Qi Liu,Jawahar Tiwari,Atul J. Butte,Vivek A. Rudrapatna

DOI: https://doi.org/10.1002/cpt.3226

2024-03-08

Clinical Pharmacology & Therapeutics

Abstract:Outpatient clinical notes are a rich source of information regarding drug safety. However, data in these notes are currently underutilized for pharmacovigilance due to methodological limitations in text mining. Large language models (LLMs) like Bidirectional Encoder Representations from Transformers (BERT) have shown progress in a range of natural language processing tasks but have not yet been evaluated on adverse event (AE) detection. We adapted a new clinical LLM, University of California – San Francisco (UCSF)‐BERT, to identify serious AEs (SAEs) occurring after treatment with a non‐steroid immunosuppressant for inflammatory bowel disease (IBD). We compared this model to other language models that have previously been applied to AE detection. We annotated 928 outpatient IBD notes corresponding to 928 individual patients with IBD for all SAE‐associated hospitalizations occurring after treatment with a non‐steroid immunosuppressant. These notes contained 703 SAEs in total, the most common of which was failure of intended efficacy. Out of eight candidate models, UCSF‐BERT achieved the highest numerical performance on identifying drug‐SAE pairs from this corpus (accuracy 88–92%, macro F1 61–68%), with 5–10% greater accuracy than previously published models. UCSF‐BERT was significantly superior at identifying hospitalization events emergent to medication use ( P < 0.01). LLMs like UCSF‐BERT achieve numerically superior accuracy on the challenging task of SAE detection from clinical notes compared with prior methods. Future work is needed to adapt this methodology to improve model performance and evaluation using multicenter data and newer architectures like Generative pre‐trained transformer (GPT). Our findings support the potential value of using large language models to enhance pharmacovigilance.

pharmacology & pharmacy
Assessing Retrieval-Augmented Large Language Model Performance in Emergency Department ICD-10-CM Coding Compared to Human Coders

Eyal Klang,Idit Tessler,Donald U Apakama,Ethan Abbott,Benjamin S Glicksberg,Monique Arnold,Akini Moses,Ankit Sakhuja,Ali Soroush,Alexander W Charney,David L Reich,Jolion McGreevy,Nicholas Gavin,Brendan Carr,Robert Freeman,Girish N Nadkarni

DOI: https://doi.org/10.1101/2024.10.15.24315526

2024-10-17

Abstract:Background: Accurate medical coding is essential for clinical and administrative purposes but complicated, time-consuming, and biased. This study compares Retrieval-Augmented Generation (RAG)-enhanced LLMs to provider-assigned codes in producing ICD-10-CM codes from emergency department (ED) clinical records. Methods: Retrospective cohort study using 500 ED visits randomly selected from the Mount Sinai Health System between January and April 2024. The RAG system integrated past 1,038,066 ED visits data (2021-2023) into the LLMs predictions to improve coding accuracy. Nine commercial and open-source LLMs were evaluated. The primary outcome was a head-to-head comparison of the ICD-10-CM codes generated by the RAG-enhanced LLMs and those assigned by the original providers. A panel of four physicians and two LLMs blindly reviewed the codes, comparing the RAG-enhanced LLM and provider-assigned codes on accuracy and specificity. Findings: RAG-enhanced LLMs demonstrated superior performance to provider coders in both the accuracy and specificity of code assignments. In a targeted evaluation of 200 cases where discrepancies existed between GPT-4 and provider-assigned codes, human reviewers favored GPT-4 for accuracy in 447 instances, compared to 277 instances where providers codes were preferred (p<0.001). Similarly, GPT-4 was selected for its superior specificity in 509 cases, whereas human coders were preferred in only 181 cases (p<0.001). Smaller open-access models, such as Llama-3.1-70B, also demonstrated substantial scalability when enhanced with RAG, with 218 instances of accuracy preference compared to 90 for providers' codes. Furthermore, across all models, the exact match rate between LLM-generated and provider-assigned codes significantly improved following RAG integration, with Qwen-2-7B increasing from 0.8% to 17.6% and Gemma-2-9b-it improving from 7.2% to 26.4%. Interpretation: RAG-enhanced LLMs improve medical coding accuracy in EDs, suggesting clinical workflow applications. These findings show that generative AI can improve clinical outcomes and reduce administrative burdens.

Health Informatics
Leveraging Large Language Models in Gynecologic Oncology: A Systematic Review of Current Applications and Challenges

Aya Mudrik,Abraham Tsur,Girish Nadkarni,Orly Efros,Benjamin S Glicksberg,Shelly Soffer,Eyal Klang

DOI: https://doi.org/10.1101/2024.08.08.24311699

2024-08-09

Abstract:Rationale and Objectives: Over the past year, studies have been conducted to evaluate the performance of Large Language Models (LLMs), such as ChatGPT, in the fields of gynecologic oncology. This review aims to analyze the applications and risks associated with using LLMs in this specialized field. Materials and Methods: This systematic review was performed in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, incorporating elements from the diagnostic test accuracy extension and the CHARMS checklist for reviews of prediction models. A systematic literature search was executed on July 17, 2024, across PubMed, Web of Science, and Scopus databases. We focused on identifying original research that integrates LLMs with gynecologic oncology. We assessed the risk of bias using the adapted QUADAS-2 criteria. Results: Our search identified eight studies that met our criteria, focusing on healthcare education, clinical practice, and medical code generation. These studies revealed variability in ChatGPT's performance across different applications. It excelled in genetic testing and counseling, achieving 97% accuracy rate. However, its performance in cervical cancer prevention was less robust, with an accuracy of 83%. While one study demonstrated ChatGPT's high adherence to quality guidelines, another noted that established guidelines significantly outperformed ChatGPT's outputs. Additionally, code generation using tools like Google Bard and RoBERTa have shown potential to improve accuracy in clinical predictions and quality assurance. For example, Natural Language Processing (NLP) assisted by RoBERTa (based on Google's BERT model) has improved the prediction of residual disease in women with advanced epithelial ovarian cancer following cytoreductive surgery. Despite these advancements, challenges related to consistency, specificity, and personalization persist, underscoring the necessity for continuous enhancement of these technologies. Conclusion: LLMs demonstrate inconsistent performance in gynecologic oncology. These findings emphasize the need for continuous evaluation of these models before they are implemented clinically.

Obstetrics and Gynecology
Automatic detection of small bowel lesions with different bleeding risks based on deep learning models

Rui-Ya Zhang,Peng-Peng Qiang,Ling-Jun Cai,Tao Li,Yan Qin,Yu Zhang,Yi-Qing Zhao,Jun-Ping Wang

DOI: https://doi.org/10.3748/wjg.v30.i2.170

IF: 5.374

2024-01-14

World Journal of Gastroenterology

Abstract:BACKGROUND Deep learning provides an efficient automatic image recognition method for small bowel (SB) capsule endoscopy (CE) that can assist physicians in diagnosis. However, the existing deep learning models present some unresolved challenges. AIM To propose a novel and effective classification and detection model to automatically identify various SB lesions and their bleeding risks, and label the lesions accurately so as to enhance the diagnostic efficiency of physicians and the ability to identify high-risk bleeding groups. METHODS The proposed model represents a two-stage method that combined image classification with object detection. First, we utilized the improved ResNet-50 classification model to classify endoscopic images into SB lesion images, normal SB mucosa images, and invalid images. Then, the improved YOLO-V5 detection model was utilized to detect the type of lesion and its risk of bleeding, and the location of the lesion was marked. We constructed training and testing sets and compared model-assisted reading with physician reading. RESULTS The accuracy of the model constructed in this study reached 98.96%, which was higher than the accuracy of other systems using only a single module. The sensitivity, specificity, and accuracy of the model-assisted reading detection of all images were 99.17%, 99.92%, and 99.86%, which were significantly higher than those of the endoscopists’ diagnoses. The image processing time of the model was 48 ms/image, and the image processing time of the physicians was 0.40 ± 0.24 s/image (P < 0.001). CONCLUSION The deep learning model of image classification combined with object detection exhibits a satisfactory diagnostic effect on a variety of SB lesions and their bleeding risks in CE images, which enhances the diagnostic efficiency of physicians and improves the ability of physicians to identify high-risk bleeding groups.

gastroenterology & hepatology
Emerging Applications of NLP and Large Language Models in Gastroenterology and Hepatology: A Systematic Review

Mahmud Omar Sr.,Kassem Sharif Sr.,Benjamin S Glicksberg Sr.,Girish Nadkarni,Eyal Klang Sr.

DOI: https://doi.org/10.1101/2024.06.26.24309567

2024-06-27

Abstract:Background and Aim: In the last two years, natural language processing (NLP) has transformed significantly with the introduction of large language models (LLM). This review updates on NLP and LLM applications and challenges in gastroenterology and hepatology. Methods: Registered with PROSPERO (CRD42024542275) and adhering to PRISMA guidelines, we searched six databases for relevant studies published from 2003 to 2024, ultimately including 57 studies. Results: Our review notes an increase in relevant publications in 2023-2024 compared to previous years, reflecting growing interest in newer models such as GPT-3 and GPT-4. The results demonstrate that NLP models have enhanced data extraction from electronic health records and other unstructured medical data sources. Key findings include high precision in identifying disease characteristics from unstructured reports and ongoing improvement in clinical decision-making. Risk of bias assessments using ROBINS-I, QUADAS-2, and PROBAST tools confirmed the methodological robustness of the included studies. Conclusion: NLP and LLMs can enhance diagnosis and treatment in gastroenterology and hepatology. They enable extraction of data from unstructured medical records, such as endoscopy reports and patient notes, and for enhancing clinical decision-making. Despite these advancements, integrating these tools into routine practice is still challenging. Future work should prospectively demonstrate real-world value. Keywords: Natural Language Processing, Large Language Models, Gastroenterology, Hepatology, Electronic Health Records.

Gastroenterology
Application of machine learning to predict postoperative gastrointestinal bleed in bariatric surgery

Justin L. Hsu,Kevin A. Chen,Logan R. Butler,Anoosh Bahraini,Muneera R. Kapadia,Shawn M. Gomez,Timothy M. Farrell

DOI: https://doi.org/10.1007/s00464-023-10156-0

2023-06-15

Surgical Endoscopy

Abstract:Postoperative gastrointestinal bleeding (GIB) is a rare but serious complication of bariatric surgery. The recent rise in extended venous thromboembolism regimens as well as outpatient bariatric surgery may increase the risk of postoperative GIB or lead to delay in diagnosis. This study seeks to use machine learning (ML) to create a model that predicts postoperative GIB to aid surgeon decision-making and improve patient counseling for postoperative bleeds.

surgery
Sex differences in vasoactive intestinal peptide (VIP) concentrations in the anterior pituitary and hypothalamus of rats

P. Riskind,Janet M. Allen,S. Gabriel,J. Koenig,Judith Audet-Arnold

DOI: https://doi.org/10.1016/0304-3940(89)90040-2

IF: 3.197

1989-10-23

Neuroscience Letters

Abstract:
An interpretable artificial intelligence system for detecting risk factors of gastroesophageal variceal bleeding

Jing Wang,Zhengqiang Wang,Mingkai Chen,Yong Xiao,Shi Chen,Lianlian Wu,Liwen Yao,Xiaoda Jiang,Jiao Li,Ming Xu,Mengjuan Lin,Yijie Zhu,Renquan Luo,Chenxia Zhang,Xun Li,Honggang Yu

DOI: https://doi.org/10.1038/s41746-022-00729-z

IF: 15.2

2022-12-19

npj Digital Medicine

Abstract:Bleeding risk factors for gastroesophageal varices (GEV) detected by endoscopy in cirrhotic patients determine the prophylactical treatment patients will undergo in the following 2 years. We propose a methodology for measuring the risk factors. We create an artificial intelligence system (ENDOANGEL-GEV) containing six models to segment GEV and to classify the grades (grades 1–3) and red color signs (RC, RC0-RC3) of varices. It also summarizes changes in the above results with region in real time. ENDOANGEL-GEV is trained using 6034 images from 1156 cirrhotic patients across three hospitals (dataset 1) and validated on multicenter datasets with 11009 images from 141 videos (dataset 2) and in a prospective study recruiting 161 cirrhotic patients from Renmin Hospital of Wuhan University (dataset 3). In dataset 1, ENDOANGEL-GEV achieves intersection over union values of 0.8087 for segmenting esophageal varices and 0.8141 for gastric varices. In dataset 2, the system maintains fairly accuracy across images from three hospitals. In dataset 3, ENDOANGEL-GEV surpasses attended endoscopists in detecting RC of GEV and classifying grades ( p < 0.001). When ranking the risk of patients combined with the Child‒Pugh score, ENDOANGEL-GEV outperforms endoscopists for esophageal varices ( p < 0.001) and shows comparable performance for gastric varices ( p = 0.152). Compared with endoscopists, ENDOANGEL-GEV may help 12.31% (16/130) more patients receive the right intervention. We establish an interpretable system for the endoscopic diagnosis and risk stratification of GEV. It will assist in detecting the first bleeding risk factors accurately and expanding the scope of quantitative measurement of diseases.

health care sciences & services,medical informatics
Predictive and Explainable Analysis of Post-operative Acute Kidney Injury in Children undergoing Cardiopulmonary Bypass: An Application of Large Language Models

Mansour T.A. Sharabiani,Alireza S Mahani,Alex Bottle,Yadav Srinivasan,Richard W Issitt,Serban Stoica

DOI: https://doi.org/10.1101/2024.05.14.24307372

2024-05-17

MedRxiv

Abstract:Motivation: Embedding large language models (LLMs) hold promise for predictive analytics due to their natively consistent numeric output. This study explores the utility of general-purpose LLMs for extracting interpretable and actionable insights from electronic health record (EHR) text columns in tabular data, focusing on paediatric cardiopulmonary bypass (CPB). Methods: We analysed data from 963 paediatric CPB operations in the UK (2019-2021), focusing on the severity of post-operative acute kidney injury (AKI) as a binary outcome, and using text columns documenting planned surgical procedures and patient diagnoses for each operation as features. We employed OpenAI's "text-embedding-3-large" for embeddings and "gpt-4-turbo" for generating descriptive labels of patient clusters that were formed by applying spherical k-means to embedding vectors. These "AI clusters" were compared against the "expert clusters" - based on the Partial Risk Adjustment in Surgery (PRAiS) v2 protocol - for 1) consistency using adjusted rand index (ARI) between the clusters and 2) predictive performance using area under ROC (AUC) of logistic regression models using cluster memberships as a categorical variable. Results: AI clusters showed statistically significant levels of consistency with expert clusters, evidenced by their ARIs of 0.31 and 0.32 for planned procedures and diagnoses, respectively. The clusters demonstrated comparable discriminative power in predicting severe post-operative kidney injury, with AUCs of 0.63 vs. 0.60 (planned procedures) and 0.56 vs. 0.58 (diagnoses) for AI clusters vs. expert clusters. Notably, AI clusters identified three interventions with significant odds ratios for AKI severity, highlighting potential areas for clinical focus. Replacing k-means clustering followed by logistic regression with k-nearest-neighbours - applied to LLM text embeddings - further improved AUC to 0.66 and 0.63, for planned procedures and diagnoses, respectively. Conclusions: Our findings affirm the potential of LLMs as effective tools in medical text analysis, facilitating both exploratory and predictive tasks. The integration of LLM-derived insights with traditional data analysis methods can significantly enhance risk stratification and outcome prediction in paediatric CPB, underscoring the value of AI-driven approaches in complex healthcare datasets.
Privacy-preserving large language models for structured medical information retrieval

Isabella Catharina Wiest,Dyke Ferber,Jiefu Zhu,Marko van Treeck,Sonja K. Meyer,Radhika Juglan,Zunamys I. Carrero,Daniel Paech,Jens Kleesiek,Matthias P. Ebert,Daniel Truhn,Jakob Nikolas Kather

DOI: https://doi.org/10.1038/s41746-024-01233-2

IF: 15.2

2024-09-21

npj Digital Medicine

Abstract:Most clinical information is encoded as free text, not accessible for quantitative analysis. This study presents an open-source pipeline using the local large language model (LLM) "Llama 2" to extract quantitative information from clinical text and evaluates its performance in identifying features of decompensated liver cirrhosis. The LLM identified five key clinical features in a zero- and one-shot manner from 500 patient medical histories in the MIMIC IV dataset. We compared LLMs of three sizes and various prompt engineering approaches, with predictions compared against ground truth from three blinded medical experts. Our pipeline achieved high accuracy, detecting liver cirrhosis with 100% sensitivity and 96% specificity. High sensitivities and specificities were also yielded for detecting ascites (95%, 95%), confusion (76%, 94%), abdominal pain (84%, 97%), and shortness of breath (87%, 97%) using the 70 billion parameter model, which outperformed smaller versions. Our study successfully demonstrates the capability of locally deployed LLMs to extract clinical information from free text with low hardware requirements.

health care sciences & services,medical informatics
[Efficacy of laparoscopy in the diagnosis of ascites of indeterminate origin].

J. García‐Pagán,J. Bordas,E. Saperas,F. Mondelo,X. Calvet,J. Rodés

1988-02-20

Abstract:
Large Language Models Improve the Identification of Emergency Department Visits for Symptomatic Kidney Stones

Cosmin A Bejan,Amy M Reed,Matthew Mikula,Siwei Zhang,Yaomin Xu,Daniel Fabbri,Peter J Embi,Ryan S Hsi

DOI: https://doi.org/10.1101/2024.08.12.24311870

2024-08-13

Abstract: Recent advancements of large language models (LLMs) like Generative Pre-trained Transformer 4 (GPT-4) have generated significant interest among the scientific community. Yet, the potential of these models to be utilized in clinical settings remains largely unexplored. This study investigated the abilities of multiple LLMs and traditional machine learning models to analyze emergency department (ED) reports and determine if the corresponding visits were caused by symptomatic kidney stones. Leveraging a dataset of manually annotated ED reports, we developed strategies to enhance the performance of GPT-4, GPT-3.5, and Llama-2 including prompt optimization, zero- and few-shot prompting, fine-tuning, and prompt augmentation. Further, we implemented fairness assessment and bias mitigation methods to investigate the potential disparities by these LLMs with respect to race and gender. A clinical expert manually assessed the explanations generated by GPT-4 for its predictions to determine if they were sound, factually correct, unrelated to the input prompt, or potentially harmful. The evaluation includes a comparison between LLMs, traditional machine learning models (logistic regression, extreme gradient boosting, and light gradient boosting machine), and a baseline system utilizing International Classification of Diseases (ICD) codes for kidney stones. The best results were achieved by GPT-4 (macro-F1=0.833, 95% confidence interval [CI]=0.826-0.841) and GPT-3.5 (macro-F1=0.796, 95% CI=0.796-0.796), both being statistically significantly better than the ICD-based baseline result (macro-F1=0.71). Ablation studies revealed that the initial pre-trained GPT-3.5 model benefits from fine-tuning when using the same parameter configuration. Adding demographic information and prior disease history to the prompts allows LLMs to make more accurate decisions. The evaluation of bias found that GPT-4 exhibited no racial or gender disparities, in contrast to GPT-3.5, which failed to effectively model racial diversity. The analysis of explanations provided by GPT-4 demonstrates advanced capabilities of this model in understanding clinical text and reasoning with medical knowledge.

Health Informatics

A strategy to improve the health of children and young people.

A Large Language Model Screening Tool to Target Patients for Best Practice Alerts: Development and Validation

Large language models for extracting histopathologic diagnoses from electronic health records

Early identification of patients with acute gastrointestinal bleeding using natural language processing and decision rules

Large language models for accurate disease detection in electronic health records

Artificial Intelligence in Gastrointestinal Bleeding Analysis for Video Capsule Endoscopy: Insights, Innovations, and Prospects (2008-2023)

Optimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes

Critical Care Studies Using Large Language Models Based on Electronic Healthcare Records: A Technical Note

Automated Multimodal Machine Learning for Esophageal Variceal Bleeding Prediction Based on Endoscopy and Structured Data

Algorithmic Identification of Treatment‐Emergent Adverse Events From Clinical Notes Using Large Language Models: A Pilot Study in Inflammatory Bowel Disease

Assessing Retrieval-Augmented Large Language Model Performance in Emergency Department ICD-10-CM Coding Compared to Human Coders

Leveraging Large Language Models in Gynecologic Oncology: A Systematic Review of Current Applications and Challenges

Automatic detection of small bowel lesions with different bleeding risks based on deep learning models

Emerging Applications of NLP and Large Language Models in Gastroenterology and Hepatology: A Systematic Review

Application of machine learning to predict postoperative gastrointestinal bleed in bariatric surgery

Sex differences in vasoactive intestinal peptide (VIP) concentrations in the anterior pituitary and hypothalamus of rats

An interpretable artificial intelligence system for detecting risk factors of gastroesophageal variceal bleeding

Predictive and Explainable Analysis of Post-operative Acute Kidney Injury in Children undergoing Cardiopulmonary Bypass: An Application of Large Language Models

Privacy-preserving large language models for structured medical information retrieval

[Efficacy of laparoscopy in the diagnosis of ascites of indeterminate origin].

Large Language Models Improve the Identification of Emergency Department Visits for Symptomatic Kidney Stones