Abstract:BACKGROUND: Medication and adverse drug event (ADE) information extracted from electronic health record (EHR) notes can be a rich resource for drug safety surveillance. Existing observational studies have mainly relied on structured EHR data to obtain ADE information; however, ADEs are often buried in the EHR narratives and not recorded in structured data.OBJECTIVE: To unlock ADE-related information from EHR narratives, there is a need to extract relevant entities and identify relations among them. In this study, we focus on relation identification. This study aimed to evaluate natural language processing and machine learning approaches using the expert-annotated medical entities and relations in the context of drug safety surveillance, and investigate how different learning approaches perform under different configurations.METHODS: We have manually annotated 791 EHR notes with 9 named entities (eg, medication, indication, severity, and ADEs) and 7 different types of relations (eg, medication-dosage, medication-ADE, and severity-ADE). Then, we explored 3 supervised machine learning systems for relation identification: (1) a support vector machines (SVM) system, (2) an end-to-end deep neural network system, and (3) a supervised descriptive rule induction baseline system. For the neural network system, we exploited the state-of-the-art recurrent neural network (RNN) and attention models. We report the performance by macro-averaged precision, recall, and F1-score across the relation types.RESULTS: Our results show that the SVM model achieved the best average F1-score of 89.1% on test data, outperforming the long short-term memory (LSTM) model with attention (F1-score of 65.72%) as well as the rule induction baseline system (F1-score of 7.47%) by a large margin. The bidirectional LSTM model with attention achieved the best performance among different RNN models. With the inclusion of additional features in the LSTM model, its performance can be boosted to an average F1-score of 77.35%.CONCLUSIONS: It shows that classical learning models (SVM) remains advantageous over deep learning models (RNN variants) for clinical relation identification, especially for long-distance intersentential relations. However, RNNs demonstrate a great potential of significant improvement if more training data become available. Our work is an important step toward mining EHRs to improve the efficacy of drug safety surveillance. Most importantly, the annotated data used in this study will be made publicly available, which will further promote drug safety research in the community.

Linking Medications and Their Attributes in Clinical Notes and Clinical Trial Announcements for Information Extraction: A Sequence Labeling Approach

A sequence labeling approach to link medications and their attributes in clinical notes and clinical trial announcements for information extraction

Applying a Deep Learning-Based Sequence Labeling Approach to Detect Attributes of Medical Concepts in Clinical Text

Identifying relations of medications with adverse drug events using recurrent convolutional neural networks and gradient boosting

MADEx: A System for Detecting Medications, Adverse Drug Events, and Their Relations from Clinical Notes

Extracting Drug Names and Associated Attributes From Discharge Summaries: Text Mining Study

A Study of Deep Learning Approaches for Medication and Adverse Drug Event Extraction from Clinical Text.

INSIGHTBUDDY-AI: Medication Extraction and Entity Linking using Large Language Models and Ensemble Learning

Clinical Relation Extraction Toward Drug Safety Surveillance Using Electronic Health Record Narratives: Classical Learning Versus Deep Learning

Medical subdomain classification of clinical notes using a machine learning-based natural language processing approach

A Study of Machine-Learning-based Approaches to Extract Clinical Entities and Their Assertions from Discharge Summaries

Automatic Extraction of Medication Data from Semi-Structured Prescriptions

Extraction of Medication and Temporal Relation from Clinical Text using Neural Language Models

Adverse drug events and medication relation extraction in electronic health records with ensemble deep learning methods

Hybrid X-Linker: Automated Data Generation and Extreme Multi-label Ranking for Biomedical Entity Linking

Mining FDA drug labels for medical conditions

Feature Engineering for Drug Name Recognition in Biomedical Texts: Feature Conjunction and Feature Selection

An investigation of single-domain and multidomain medication and adverse drug event relation extraction from electronic health record notes using advanced deep learning models

Extracting social determinants of health from clinical note text with classification and sequence-to-sequence approaches

Feature Engineering Combined with Machine Learning and Rule-Based Methods for Structured Information Extraction from Narrative Clinical Discharge Summaries

Weakly Supervised Medication Regimen Extraction from Medical Conversations