Abstract:Background: Rich data in cardiovascular diagnostic testing is often sequestered in unstructured reports, with the necessity of manual abstraction limiting their use in real-time applications in patient care and research. Methods: We developed a two-step process that sequentially deploys generative and interpretative large language models (LLMs; Llama2 70b and Llama2 13b). Using a Llama2 70b model, we generated varying formats of transthoracic echocardiogram (TTE) reports from 3000 real-world echo reports with paired structured elements, leveraging temporal changes in reporting formats to define the variations. Subsequently, we fine-tuned Llama2 13b using sequentially larger batches of generated echo reports as inputs, to extract data from free-text narratives across 18 clinically relevant echocardiographic fields. This was set up as a prompt-based supervised training task. We evaluated the fine-tuned Llama2 13b model, HeartDx-LM, on several distinct echocardiographic datasets: (i) reports across the different time periods and formats at Yale New Haven Health System (YNHHS), (ii) the Medical Information Mart for Intensive Care (MIMIC) III dataset, and (iii) the MIMIC IV dataset. We used the accuracy of extracted fields and Cohen's Kappa as the metrics and have publicly released the HeartDX-LM model. Results: The HeartDX-LM model was trained on randomly selected 2,000 synthetic echo reports with varying formats and paired structured labels, with a wide range of clinical findings. We identified a lower threshold of 500 annotated reports required for fine-tuning Llama2 13b to achieve stable and consistent performance. At YNHHS, the HeartDx-LM model accurately extracted 69,144 out of 70,032 values (98.7%) across 18 clinical fields from unstructured reports in the test set from contemporary records where paired structured data were also available. In older echo reports where only unstructured reports were available, the model achieved 87.1% accuracy against expert annotations for the same 18 fields for a random sample of 100 reports. Similarly, in expert-annotated external validation sets from MIMIC-IV and MIMIC-III, HeartDx-LM correctly extracted 201 out of 220 available values (91.3%) and 615 out of 707 available values (87.9%), respectively, from 100 randomly chosen and expert annotated echo reports from each set. Conclusion: We developed a novel method using paired large and moderate-sized LLMs to automate the extraction of unstructured echocardiographic reports into tabular datasets. Our approach represents a scalable strategy that transforms unstructured reports into computable elements that can be leveraged to improve cardiovascular care quality and enable research.

A novel heterocyclic compound: catena-poly[[[diaquasodium(I)]-di-mu-aqua] hemi(1,5-dihydroxy-4,8,9-trioxa-2,6-diazabicyclo[3.3.1]nona-2,6-diene-3,7-diolate)].

Implementing engineering education on the WWW: 3 case studies

Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models

Precision measurement of cardiac structure and function in cardiovascular magnetic resonance using machine learning

Automated Transformation of Unstructured Cardiovascular Diagnostic Reports into Structured Datasets Using Sequentially Deployed Large Language Models

An AI tool for automated analysis of large-scale unstructured clinical cine CMR databases

Empirical evaluation of language modeling to ascertain cancer outcomes from clinical text reports

Leukemia among uranium miners--late effects of exposure to uranium dust?

Longitudinal Multimodal Transformer Integrating Imaging and Latent Clinical Signatures From Routine EHRs for Pulmonary Nodule Classification

Multi-objective Representation for Numbers in Clinical Narratives Using CamemBERT-bio

Ultrasound investigation prior to organ donation.

Numerical Attributes Learning for Cardiac Failure Diagnostic from Clinical Narratives -- A LESA-CamemBERT-bio Approach

Clinical assessment of an AI tool for measuring biventricular parameters on cardiac MR

A Comparative Study of Pretrained Language Models for Long Clinical Text

Deep Learning Analysis of Cardiac MRI in Legacy Datasets: Multi-Ethnic Study of Atherosclerosis

Identify diabetic retinopathy-related clinical concepts and their attributes using transformer-based natural language processing methods

SAT Modulo Theories : Can we get the best of two worlds ? Invited talk , CP 2010-St Andrews

Fusing Echocardiography Images and Medical Records for Continuous Patient Stratification

Nested-polymerase chain reaction for the detection of Helicobacter pylori infection with novel primers designed by sequence analysis of urease A gene in clinically isolated bacterial strains.

Deep learning models for triaging hospital head MRI examinations

Early Rehabilitation in Elderly after Arthroplasty versus Internal Fixation for Unstable Intertrochanteric Fractures of Femur: Systematic Review and Meta-Analysis