Abstract:Background: Cardiac magnetic resonance imaging (CMR) is a powerful diagnostic modality that provides detailed quantitative assessment of cardiac anatomy and function. Automated extraction of CMR measurements from clinical reports that are typically stored as unstructured text in electronic health record systems would facilitate their use in research. Existing machine learning approaches either rely on large quantities of expert annotation or require the development of engineered rules that are time-consuming and are specific to the setting in which they were developed. Objective: We hypothesize that the use of pretrained transformer-based language models may enable label-efficient numerical extraction from clinical text without the need for heuristics or large quantities of expert annotations. Here, we fine-tuned pretrained transformer-based language models on a small quantity of CMR annotations to extract 21 CMR measurements. We assessed the effect of clinical pretraining to reduce labeling needs and explored alternative representations of numerical inputs to improve performance. Methods: Our study sample comprised 99,252 patients that received longitudinal cardiology care in a multi-institutional health care system. There were 12,720 available CMR reports from 9280 patients. We adapted PRAnCER (Platform Enabling Rapid Annotation for Clinical Entity Recognition), an annotation tool for clinical text, to collect annotations from a study clinician on 370 reports. We experimented with 5 different representations of numerical quantities and several model weight initializations. We evaluated extraction performance using macroaveraged F1-scores across the measurements of interest. We applied the best-performing model to extract measurements from the remaining CMR reports in the study sample and evaluated established associations between selected extracted measures with clinical outcomes to demonstrate validity. Results: All combinations of weight initializations and numerical representations obtained excellent performance on the gold-standard test set, suggesting that transformer models fine-tuned on a small set of annotations can effectively extract numerical quantities. Our results further indicate that custom numerical representations did not appear to have a significant impact on extraction performance. The best-performing model achieved a macroaveraged F1-score of 0.957 across the evaluated CMR measurements (range 0.92 for the lowest-performing measure of left atrial anterior-posterior dimension to 1.0 for the highest-performing measures of left ventricular end systolic volume index and left ventricular end systolic diameter). Application of the best-performing model to the study cohort yielded 136,407 measurements from all available reports in the study sample. We observed expected associations between extracted left ventricular mass index, left ventricular ejection fraction, and right ventricular ejection fraction with clinical outcomes like atrial fibrillation, heart failure, and mortality. Conclusions: This study demonstrated that a domain-agnostic pretrained transformer model is able to effectively extract quantitative clinical measurements from diagnostic reports with a relatively small number of gold-standard annotations. The proposed workflow may serve as a roadmap for other quantitative entity extraction.

A Novel Sentence Transformer-based Natural Language Processing Approach for Schema Mapping of Electronic Health Records to the OMOP Common Data Model

Measurement of Semantic Textual Similarity in Clinical Texts: Comparison of Transformer-Based Models

OpenDeID Pipeline for Unstructured Electronic Health Record Text Notes Based on Rules and Transformers: Deidentification Algorithm Development and Validation Study

Developing a natural language processing system using transformer-based models for adverse drug event detection in electronic health records

CDEMapper: Enhancing NIH Common Data Element Normalization using Large Language Models

Automatic Conversion of Electronic Medical Record Text for OpenEHR Based on Semantic Analysis

Extracting Family History of Patients from Clinical Narratives: Exploring an End-to-End Solution with Deep Learning Models.

Transformers and large language models in healthcare: A review

Leveraging Open-Source Large Language Models for encoding Social Determinants of Health using an Intelligent Router

Transformer-based Named Entity Recognition for Parsing Clinical Trial Eligibility Criteria

Improving Extraction of Clinical Event Contextual Properties from Electronic Health Records: A Comparative Study

Structuring Electronic Dental Records Through Deep Learning for a Clinical Decision Support System.

One Clinician Is All You Need-Cardiac Magnetic Resonance Imaging Measurement Extraction: Deep Learning Algorithm Development

EHRmonize: A Framework for Medical Concept Abstraction from Electronic Health Records using Large Language Models

Healthcare NER Models Using Language Model Pretraining

Identification of Dietary Supplement Use from Electronic Health Records Using Transformer-based Language Models

Identify diabetic retinopathy-related clinical concepts and their attributes using transformer-based natural language processing methods

Automatic Data Transformation Using Large Language Model: An Experimental Study on Building Energy Data

Coupling Symbolic Reasoning with Language Modeling for Efficient Longitudinal Understanding of Unstructured Electronic Medical Records

Med7: a transferable clinical natural language processing model for electronic health records

MedT5SQL: a transformers-based large language model for text-to-SQL conversion in the healthcare domain