[Flourescent amoebic antibody test in a patient with secondary carcinoma of the liver (author's transl)].

J. Bompart,J. Gendre,P. Romano,R. Houin,Y. Le Quintrec

Abstract:

What problem does this paper attempt to address?

Evaluating robustness of language models for chief complaint extraction from patient-generated text

Ilya Valmianski,Caleb Goodwin,Ian M. Finn,Naqi Khan,Daniel S. Zisook

DOI: https://doi.org/10.48550/arXiv.1911.06915

2019-11-16

Abstract:Automated classification of chief complaints from patient-generated text is a critical first step in developing scalable platforms to triage patients without human intervention. In this work, we evaluate several approaches to chief complaint classification using a novel Chief Complaint (CC) Dataset that contains ~200,000 patient-generated reasons-for-visit entries mapped to a set of 795 discrete chief complaints. We examine the use of several fine-tuned bidirectional transformer (BERT) models trained on both unrelated texts as well as on the CC dataset. We contrast this performance with a TF-IDF baseline. Our evaluation has three components: (1) a random test hold-out from the original dataset; (2) a "misspelling set," consisting of a hand-selected subset of the test set, where every entry has at least one misspelling; (3) a separate experimenter-generated free-text set. We find that the TF-IDF model performs significantly better than the strongest BERT-based model on the test (best BERT PR-AUC $0.3597 \pm 0.0041$ vs TF-IDF PR-AUC $0.3878 \pm 0.0148$, $p=7\cdot 10^{-5}$), and is statistically comparable to the misspelling sets (best BERT PR-AUC $0.2579 \pm 0.0079$ vs TF-IDF PR-AUC $0.2733 \pm 0.0130$, $p=0.06$). However, when examining model predictions on experimenter-generated queries, some concerns arise about TF-IDF baseline's robustness. Our results suggest that in certain tasks, simple language embedding baselines may be very performant; however, truly understanding their robustness requires further analysis.

Computation and Language
Using a clinical narrative-aware pre-trained language model for predicting emergency department patient disposition and unscheduled return visits

Tzu-Ying Chen,Ting-Yun Huang,Yung-Chun Chang

DOI: https://doi.org/10.1016/j.jbi.2024.104657

Abstract:The increasing prevalence of overcrowding in Emergency Departments (EDs) threatens the effective delivery of urgent healthcare. Mitigation strategies include the deployment of monitoring systems capable of tracking and managing patient disposition to facilitate appropriate and timely care, which subsequently reduces patient revisits, optimizes resource allocation, and enhances patient outcomes. This study used ∼ 250,000 emergency department visit records from Taipei Medical University-Shuang Ho Hospital to develop a natural language processing model using BlueBERT, a biomedical domain-specific pre-trained language model, to predict patient disposition status and unplanned readmissions. Data preprocessing and the integration of both structured and unstructured data were central to our approach. Compared to other models, BlueBERT outperformed due to its pre-training on a diverse range of medical literature, enabling it to better comprehend the specialized terminology, relationships, and context present in ED data. We found that translating Chinese-English clinical narratives into English and textualizing numerical data into categorical representations significantly improved the prediction of patient disposition (AUROC = 0.9014) and 72-hour unscheduled return visits (AUROC = 0.6475). The study concludes that the BlueBERT-based model demonstrated superior prediction capabilities, surpassing the performance of prior patient disposition predictive models, thus offering promising applications in the realm of ED clinical practice.
Manganese Reduction by Microbes from Oxic Regions of the Lake Vanda (Antarctica) Water Column

B. Bratina,B. Stevenson,W. Green,T. Schmidt

DOI: https://doi.org/10.1128/AEM.64.10.3791-3797.1998

1998-10-01

Applied and Environmental Microbiology

Abstract:ABSTRACT Depth profiles of metals in Lake Vanda, a permanently ice-covered, stratified Antarctic lake, suggest the importance of particulate manganese oxides in the scavenging, transport, and release of metals. Since manganese oxides can be solubilized by manganese-reducing bacteria, microbially mediated manganese reduction was investigated in Lake Vanda. Microbes concentrated from oxic regions of the water column, encompassing a peak of soluble manganese [Mn(II)], reduced synthetic manganese oxides (MnO2) when incubated aerobically. Pure cultures of manganese-reducing bacteria were readily isolated from waters collected near the oxic Mn(II) peak. Based on phylogenetic analysis of the 16S rRNA gene sequence, most of the isolated manganese reducers belong to the genusCarnobacterium. Cultures of a phylogenetically representative strain of Carnobacterium reduced synthetic MnO2 in the presence of sodium azide, as was seen in field assays. Unlike anaerobes that utilize manganese oxides as terminal electron acceptors in respiration, isolates of the genusCarnobacterium reduced Mn(IV) via a diffusible compound under oxic conditions. The release of adsorbed trace metals accompanying the solubilization of manganese oxides may provide populations of Carnobacterium with a source of nutrients in this extremely oligotrophic environment.
Autocompletion of Chief Complaints in the Electronic Health Records using Large Language Models

K M Sajjadul Islam,Ayesha Siddika Nipu,Praveen Madiraju,Priya Deshpande

2024-01-12

Abstract:The Chief Complaint (CC) is a crucial component of a patient's medical record as it describes the main reason or concern for seeking medical care. It provides critical information for healthcare providers to make informed decisions about patient care. However, documenting CCs can be time-consuming for healthcare providers, especially in busy emergency departments. To address this issue, an autocompletion tool that suggests accurate and well-formatted phrases or sentences for clinical notes can be a valuable resource for triage nurses. In this study, we utilized text generation techniques to develop machine learning models using CC data. In our proposed work, we train a Long Short-Term Memory (LSTM) model and fine-tune three different variants of Biomedical Generative Pretrained Transformers (BioGPT), namely microsoft/biogpt, microsoft/BioGPT-Large, and microsoft/BioGPT-Large-PubMedQA. Additionally, we tune a prompt by incorporating exemplar CC sentences, utilizing the OpenAI API of GPT-4. We evaluate the models' performance based on the perplexity score, modified BERTScore, and cosine similarity score. The results show that BioGPT-Large exhibits superior performance compared to the other models. It consistently achieves a remarkably low perplexity score of 1.65 when generating CC, whereas the baseline LSTM model achieves the best perplexity score of 170. Further, we evaluate and assess the proposed models' performance and the outcome of GPT-4.0. Our study demonstrates that utilizing LLMs such as BioGPT, leads to the development of an effective autocompletion tool for generating CC documentation in healthcare settings.

Computation and Language,Artificial Intelligence,Machine Learning
Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing

Sifei Han,Robert F. Zhang,Lingyun Shi,Russell Richie,Haixia Liu,Andrew Tseng,Wei Quan,Neal Ryan,David Brent,Fuchiang R. Tsui

DOI: https://doi.org/10.1016/j.jbi.2021.103984

IF: 8

2022-03-01

Journal of Biomedical Informatics

Abstract:OBJECTIVE: Social determinants of health (SDOH) are non-medical factors that can profoundly impact patient health outcomes. However, SDOH are rarely available in structured electronic health record (EHR) data such as diagnosis codes, and more commonly found in unstructured narrative clinical notes. Hence, identifying social context from unstructured EHR data has become increasingly important. Yet, previous work on using natural language processing to automate extraction of SDOH from text (a) usually focuses on an ad hoc selection of SDOH, and (b) does not use the latest advances in deep learning. Our objective was to advance automatic extraction of SDOH from clinical text by (a) systematically creating a set of SDOH based on standard biomedical and psychiatric ontologies, and (b) training state-of-the-art deep neural networks to extract mentions of these SDOH from clinical notes.DESIGN: A retrospective cohort study.SETTING AND PARTICIPANTS: Data were extracted from the Medical Information Mart for Intensive Care (MIMIC-III) database. The corpus comprised 3,504 social related sentences from 2,670 clinical notes.METHODS: We developed a framework for automated classification of multiple SDOH categories. Our dataset comprised narrative clinical notes under the "Social Work" category in the MIMIC-III Clinical Database. Using standard terminologies, SNOMED-CT and DSM-IV, we systematically curated a set of 13 SDOH categories and created annotation guidelines for these. After manually annotating the 3,504 sentences, we developed and tested three deep neural network (DNN) architectures - convolutional neural network (CNN), long short-term memory (LSTM) network, and the Bidirectional Encoder Representations from Transformers (BERT) - for automated detection of eight SDOH categories. We also compared these DNNs to three baselines models: (1) cTAKES, as well as (2) L2-regularized logistic regression and (3) random forests on bags-of-words. Model evaluation metrics included micro- and macro- F1, and area under the receiver operating characteristic curve (AUC).RESULTS: All three DNN models accurately classified all SDOH categories (minimum micro-F1 = 0.632, minimum macro-AUC = 0.854). Compared to the CNN and LSTM, BERT performed best in most key metrics (micro-F1 = 0.690, macro-AUC = 0.907). The BERT model most effectively identified the "occupational" category (F1 = 0.774, AUC = 0.965) and least effectively identified the "non-SDOH" category (F = 0.491, AUC = 0.788). BERT outperformed cTAKES in distinguishing social vs non-social sentences (BERT F1 = 0.87 vs. cTAKES F1 = 0.06), and outperformed logistic regression (micro-F1 = 0.649, macro-AUC = 0.696) and random forest (micro-F1 = 0.502, macro-AUC = 0.523) trained on bag-of-words.CONCLUSIONS: Our study framework with DNN models demonstrated improved performance for efficiently identifying a systematic range of SDOH categories from clinical notes in the EHR. Improved identification of patient SDOH may further improve healthcare outcomes.

medical informatics,computer science, interdisciplinary applications
Understanding patient complaint characteristics using contextual clinical BERT embeddings

Budhaditya Saha,Sanal Lisboa,Shameek Ghosh

DOI: https://doi.org/10.1109/embc44109.2020.9175577

2020-07-01

Abstract:In clinical conversational applications, extracted entities tend to capture the main subject of a patient's complaint, namely symptoms or diseases. However, they mostly fail to recognize the characterizations of a complaint such as the time, the onset, and the severity. For example, if the input is "I have a headache and it is extreme", state-of-the-art models only recognize the main symptom entity - headache, but ignore the severity factor of extreme, that characterises headache. In this paper, we design a two-fold approach to detect the characterizations of entities like symptoms presented by general users in contexts where they would describe their symptoms to a clinician. We use Word2Vec and BERT models to encode clinical text given by the patients. We transform the output and re-frame the task as a multi-label classification problem. Finally, we combine the processed encodings with the Linear Discriminant Analysis (LDA) algorithm to classify the characterizations of the main entity. Experimental results demonstrate that our method achieves 40-50% improvement in the accuracy over the state-of-the-art models.
[Drug permeation through artificial lipoid membranes. 19. The mechanism of ion pair permeation--alkali ion countertransport].

R. Neubert,W. Fürst,H. Schleiermacher

1985-07-01

Pharmazie

Abstract:
Incorporating Domain Knowledge Into Language Models by Using Graph Convolutional Networks for Assessing Semantic Textual Similarity: Model Development and Performance Comparison

David Chang,Eric Lin,Cynthia Brandt,Richard Andrew Taylor

DOI: https://doi.org/10.2196/23101

2021-11-26

Abstract:Background: Although electronic health record systems have facilitated clinical documentation in health care, they have also introduced new challenges, such as the proliferation of redundant information through the use of copy and paste commands or templates. One approach to trimming down bloated clinical documentation and improving clinical summarization is to identify highly similar text snippets with the goal of removing such text. Objective: We developed a natural language processing system for the task of assessing clinical semantic textual similarity. The system assigns scores to pairs of clinical text snippets based on their clinical semantic similarity. Methods: We leveraged recent advances in natural language processing and graph representation learning to create a model that combines linguistic and domain knowledge information from the MedSTS data set to assess clinical semantic textual similarity. We used bidirectional encoder representation from transformers (BERT)-based models as text encoders for the sentence pairs in the data set and graph convolutional networks (GCNs) as graph encoders for corresponding concept graphs that were constructed based on the sentences. We also explored techniques, including data augmentation, ensembling, and knowledge distillation, to improve the model's performance, as measured by the Pearson correlation coefficient (r). Results: Fine-tuning the BERT_base and ClinicalBERT models on the MedSTS data set provided a strong baseline (Pearson correlation coefficients: 0.842 and 0.848, respectively) compared to those of the previous year's submissions. Our data augmentation techniques yielded moderate gains in performance, and adding a GCN-based graph encoder to incorporate the concept graphs also boosted performance, especially when the node features were initialized with pretrained knowledge graph embeddings of the concepts (r=0.868). As expected, ensembling improved performance, and performing multisource ensembling by using different language model variants, conducting knowledge distillation with the multisource ensemble model, and taking a final ensemble of the distilled models further improved the system's performance (Pearson correlation coefficients: 0.875, 0.878, and 0.882, respectively). Conclusions: This study presents a system for the MedSTS clinical semantic textual similarity benchmark task, which was created by combining BERT-based text encoders and GCN-based graph encoders in order to incorporate domain knowledge into the natural language processing pipeline. We also experimented with other techniques involving data augmentation, pretrained concept embeddings, ensembling, and knowledge distillation to further increase our system's performance. Although the task and its benchmark data set are in the early stages of development, this study, as well as the results of the competition, demonstrates the potential of modern language model-based systems to detect redundant information in clinical notes.
Multi-Label Classification in Patient-Doctor Dialogues With the RoBERTa-WWM-ext + CNN (Robustly Optimized Bidirectional Encoder Representations From Transformers Pretraining Approach With Whole Word Masking Extended Combining a Convolutional Neural Network) Model: Named Entity Study

Yuanyuan Sun,Dongping Gao,Xifeng Shen,Meiting Li,Jiale Nan,Weining Zhang

DOI: https://doi.org/10.2196/35606

2022-04-21

Abstract:Background: With the prevalence of online consultation, many patient-doctor dialogues have accumulated, which, in an authentic language environment, are of significant value to the research and development of intelligent question answering and automated triage in recent natural language processing studies. Objective: The purpose of this study was to design a front-end task module for the network inquiry of intelligent medical services. Through the study of automatic labeling of real doctor-patient dialogue text on the internet, a method of identifying the negative and positive entities of dialogues with higher accuracy has been explored. Methods: The data set used for this study was from the Spring Rain Doctor internet online consultation, which was downloaded from the official data set of Alibaba Tianchi Lab. We proposed a composite abutting joint model, which was able to automatically classify the types of clinical finding entities into the following 4 attributes: positive, negative, other, and empty. We adapted a downstream architecture in Chinese Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach (RoBERTa) with whole word masking (WWM) extended (RoBERTa-WWM-ext) combining a text convolutional neural network (CNN). We used RoBERTa-WWM-ext to express sentence semantics as a text vector and then extracted the local features of the sentence through the CNN, which was our new fusion model. To verify its knowledge learning ability, we chose Enhanced Representation through Knowledge Integration (ERNIE), original Bidirectional Encoder Representations from Transformers (BERT), and Chinese BERT with WWM to perform the same task, and then compared the results. Precision, recall, and macro-F1 were used to evaluate the performance of the methods. Results: We found that the ERNIE model, which was trained with a large Chinese corpus, had a total score (macro-F1) of 65.78290014, while BERT and BERT-WWM had scores of 53.18247117 and 69.2795315, respectively. Our composite abutting joint model (RoBERTa-WWM-ext + CNN) had a macro-F1 value of 70.55936311, showing that our model outperformed the other models in the task. Conclusions: The accuracy of the original model can be greatly improved by giving priority to WWM and replacing the word-based mask with unit to classify and label medical entities. Better results can be obtained by effectively optimizing the downstream tasks of the model and the integration of multiple models later on. The study findings contribute to the translation of online consultation information into machine-readable information.
Enhancing efficiency and capacity of telehealth services with intelligent triage: a bidirectional LSTM neural network model employing character embedding

Jinming Shi,Ming Ye,Haotian Chen,Yaoen Lu,Zhongke Tan,Zhaohan Fan,Jie Zhao

DOI: https://doi.org/10.1186/s12911-023-02367-1

2023-11-21

Abstract:Background: The widespread adoption of telehealth services necessitates accurate online department selection based on patient medical records, a task requiring significant medical knowledge. Incorrect triage results in considerable time wastage for both patients and medical professionals. To address this, we propose an intelligent triage model based on a Bidirectional Long Short-Term Memory (Bi-LSTM) neural network with character embedding to enhance the efficiency and capacity of telehealth services. Methods: We gathered a 1.3 GB medical dataset comprising 200,000 records, each including medical history, physical examination data, and other pertinent information found on the electronic medical record homepage. Following data preprocessing, a clinical corpus was established to train character embeddings with a medical context. These character embeddings were then utilized to extract features from patient chief complaints, and a 2-layer Bi-LSTM neural network was trained to categorize these complaints, enabling intelligent triage for telehealth services. Results: 60,000 chief complaint-department data pairs were extracted from clinical corpus and divided into the training, validation, and test sets of 42,000, 9,000, and 9,000, respectively. The character embedding based Bi-LSTM neural network achieved a macro-precision of 85.50% and an F1 score of 85.45%. Conclusion: The telehealth triage model developed in this study demonstrates strong implementation outcomes and significantly improves the efficiency and capacity of telehealth services. Character embedding outperforms word embedding, and future work will incorporate additional features such as patient age and gender into the chief complaint feature to future enhance model performance.
Regles positionnelles iteratives, principe majoritaire et preferences unimodales

D. Lepelley,L. Vidu

DOI: https://doi.org/10.1051/ro:2000118

2000-07-01

Abstract:Sequential scoring rules are multi-stage social choice tules that work as follows: at each stage of the process, a score is computed for each alternative by taking into account its position in the individual preference rankings, and the alternative with the lowest score is eliminated. The current paper studies the ability of these rules for choosing the Condorcet winner (or the strong Condorcet winner) when individual preferences are single-peaked.
Emergency Department Decision Support using Clinical Pseudo-notes

Simon A. Lee,Sujay Jain,Alex Chen,Kyoka Ono,Jennifer Fang,Akos Rudas,Jeffrey N. Chiang

2024-04-30

Abstract:In this work, we introduce the Multiple Embedding Model for EHR (MEME), an approach that serializes multimodal EHR tabular data into text using pseudo-notes, mimicking clinical text generation. This conversion not only preserves better representations of categorical data and learns contexts but also enables the effective employment of pretrained foundation models for rich feature representation. To address potential issues with context length, our framework encodes embeddings for each EHR modality separately. We demonstrate the effectiveness of MEME by applying it to several decision support tasks within the Emergency Department across multiple hospital systems. Our findings indicate that MEME outperforms traditional machine learning, EHR-specific foundation models, and general LLMs, highlighting its potential as a general and extendible EHR representation strategy.

Computation and Language
Leveraging Pre-trained and Transformer-derived Embeddings from EHRs to Characterize Heterogeneity Across Alzheimer's Disease and Related Dementias

Matthew West,Colin Magdamo,Lily Cheng,Yingnan He,Sudeshna Das

2024-03-31

Abstract:Alzheimer's disease is a progressive, debilitating neurodegenerative disease that affects 50 million people globally. Despite this substantial health burden, available treatments for the disease are limited and its fundamental causes remain poorly understood. Previous work has suggested the existence of clinically-meaningful sub-types, which it is suggested may correspond to distinct etiologies, disease courses, and ultimately appropriate treatments. Here, we use unsupervised learning techniques on electronic health records (EHRs) from a cohort of memory disorder patients to characterise heterogeneity in this disease population. Pre-trained embeddings for medical codes as well as transformer-derived Clinical BERT embeddings of free text are used to encode patient EHRs. We identify the existence of sub-populations on the basis of comorbidities and shared textual features, and discuss their clinical significance.

Machine Learning
Emergency department disposition prediction using a deep neural network with integrated clinical narratives and structured data

Chien-Hua Chen,Jer-Guang Hsieh,Shu-Ling Cheng,Yih-Lon Lin,Po-Hsiang Lin,Jyh-Horng Jeng

DOI: https://doi.org/10.1016/j.ijmedinf.2020.104146

Abstract:Background: Emergency department (ED) overcrowding has been a serious issue and demands effective clinical decision-making of patient disposition. In previous studies, emergency clinical narratives provide a rich context for clinical decisions. We aimed to develop the disposition prediction model using deep learning modeling strategy with the heterogeneous data, including the physicians' narratives. Methods: We constructed a retrospective cohort of all 104,083 ED visits of non-trauma adults during 2017-18 from an academically affiliated ED in Taiwan. 18,308 visits were excluded based on the completeness of each record and the unpredictable dispositions, such as out-of-hospital cardiac arrest, against-advice discharge, and escapes. We integrated subjective section of the first physicians' clinical narratives and structured data (e.g., demographics, triage vital signs, etc.) as available predictors at the first physician-patient encounter. To predict final patient disposition (i.e., hospitalization or discharge), a deep neural network (DNN) model was developed with word embedding, a common natural language processing method. We compared the proposed model to a reference model using the Rapid Emergency Medicine Score, a logistic regression model with structured data, and a DNN model with paragraph vectors. F1 score was used to measure the predictive performance for each model. Results: The F1 score (with 95 % CI) for the proposed model, the reference model, the logistic regression model with structured data, and the DNN model with paragraph vectors were 0.674 (0.669-0.679), 0.474 (0.469-0.479), 0.547 (0.543-0.551), and 0.602 (0.596-0.607), respectively. While analyzing the relationship between context length and predictive performance under the proposed model, the F1 score at 95th percentile of the word counts was higher than that at 25th percentile of the word counts in chief complaint [0.634 (0.629-0.640) vs. 0.624 (0.620-0.628)] and in present illness [0.671 (0.667-0.674) vs. 0.654 (0.651-0.658)], but not in past medical history [0.674 (0.669-0.679) vs. 0.673 (0.666-0.679)]. Conclusions: The proposed deep learning model with the usage of the first physicians' clinical narratives and structured data based on natural language processing outperformed the commonly used ones in terms of F1 score. It also evidenced the importance of the subjective section of clinical narratives, which serve as vital predictors for ED clinical decision-making.
Clinical Named Entity Recognition using Contextualized Token Representations

Yichao Zhou,Chelsea Ju,J. Harry Caufield,Kevin Shih,Calvin Chen,Yizhou Sun,Kai-Wei Chang,Peipei Ping,Wei Wang

DOI: https://doi.org/10.48550/arXiv.2106.12608

2021-06-24

Abstract:The clinical named entity recognition (CNER) task seeks to locate and classify clinical terminologies into predefined categories, such as diagnostic procedure, disease disorder, severity, medication, medication dosage, and sign symptom. CNER facilitates the study of side-effect on medications including identification of novel phenomena and human-focused information extraction. Existing approaches in extracting the entities of interests focus on using static word embeddings to represent each word. However, one word can have different interpretations that depend on the context of the sentences. Evidently, static word embeddings are insufficient to integrate the diverse interpretation of a word. To overcome this challenge, the technique of contextualized word embedding has been introduced to better capture the semantic meaning of each word based on its context. Two of these language models, ELMo and Flair, have been widely used in the field of Natural Language Processing to generate the contextualized word embeddings on domain-generic documents. However, these embeddings are usually too general to capture the proximity among vocabularies of specific domains. To facilitate various downstream applications using clinical case reports (CCRs), we pre-train two deep contextualized language models, Clinical Embeddings from Language Model (C-ELMo) and Clinical Contextual String Embeddings (C-Flair) using the clinical-related corpus from the PubMed Central. Explicit experiments show that our models gain dramatic improvements compared to both static word embeddings and domain-generic language models.

Computation and Language,Quantitative Methods
CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks

Chao Pang,Xinzhuo Jiang,Krishna S Kalluri,Matthew Spotnitz,RuiJun Chen,Adler Perotte,Karthik Natarajan

DOI: https://doi.org/10.48550/arXiv.2111.08585

2021-11-11

Abstract:Embedding algorithms are increasingly used to represent clinical concepts in healthcare for improving machine learning tasks such as clinical phenotyping and disease prediction. Recent studies have adapted state-of-the-art bidirectional encoder representations from transformers (BERT) architecture to structured electronic health records (EHR) data for the generation of contextualized concept embeddings, yet do not fully incorporate temporal data across multiple clinical domains. Therefore we developed a new BERT adaptation, CEHR-BERT, to incorporate temporal information using a hybrid approach by augmenting the input to BERT using artificial time tokens, incorporating time, age, and concept embeddings, and introducing a new second learning objective for visit type. CEHR-BERT was trained on a subset of Columbia University Irving Medical Center-York Presbyterian Hospital's clinical data, which includes 2.4M patients, spanning over three decades, and tested using 4-fold cross-validation on the following prediction tasks: hospitalization, death, new heart failure (HF) diagnosis, and HF readmission. Our experiments show that CEHR-BERT outperformed existing state-of-the-art clinical BERT adaptations and baseline models across all 4 prediction tasks in both ROC-AUC and PR-AUC. CEHR-BERT also demonstrated strong transfer learning capability, as our model trained on only 5% of data outperformed comparison models trained on the entire data set. Ablation studies to better understand the contribution of each time component showed incremental gains with every element, suggesting that CEHR-BERT's incorporation of artificial time tokens, time and age embeddings with concept embeddings, and the addition of the second learning objective represents a promising approach for future BERT-based clinical embeddings.

Machine Learning
A pre-trained language model for emergency department intervention prediction using routine physiological data and clinical narratives

Ting-Yun Huang,Chee-Fah Chong,Heng-Yu Lin,Tzu-Ying Chen,Yung-Chun Chang,Ming-Chin Lin

DOI: https://doi.org/10.1016/j.ijmedinf.2024.105564

Abstract:Introduction: The urgency and complexity of emergency room (ER) settings require precise and swift decision-making processes for patient care. Ensuring the timely execution of critical examinations and interventions is vital for reducing diagnostic errors, but the literature highlights a need for innovative approaches to optimize diagnostic accuracy and patient outcomes. In response, our study endeavors to create predictive models for timely examinations and interventions by leveraging the patient's symptoms and vital signs recorded during triage, and in so doing, augment traditional diagnostic methodologies. Methods: Focusing on four key areas-medication dispensing, vital interventions, laboratory testing, and emergency radiology exams, the study employed Natural Language Processing (NLP) and seven advanced machine learning techniques. The research was centered around the innovative use of BioClinicalBERT, a state-of-the-art NLP framework. Results: BioClinicalBERT emerged as the superior model, outperforming others in predictive accuracy. The integration of physiological data with patient narrative symptoms demonstrated greater effectiveness compared to models based solely on textual data. The robustness of our approach was confirmed by an Area Under the Receiver Operating Characteristic curve (AUROC) score of 0.9. Conclusion: The findings of our study underscore the feasibility of establishing a decision support system for emergency patients, targeting timely interventions and examinations based on a nuanced analysis of symptoms. By using an advanced natural language processing technique, our approach shows promise for enhancing diagnostic accuracy. However, the current model is not yet fully mature for direct implementation into daily clinical practice. Recognizing the imperative nature of precision in the ER environment, future research endeavors must focus on refining and expanding predictive models to include detailed timely examinations and interventions. Although the progress achieved in this study represents an encouraging step towards a more innovative and technology-driven paradigm in emergency care, full clinical integration warrants further exploration and validation.
Natural products containing a nitrogen-nitrogen bond.

Lachlan M. Blair,J. Sperry

DOI: https://doi.org/10.1021/np400124n

IF: 5.1

2013-04-11

Journal of Natural Products

Abstract:As of early 2013, over 200 natural products are known to contain a nitrogen-nitrogen (N-N) bond. This report categorizes these compounds by structural class and details their isolation and biological activity.
Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction

Laila Rasmy,Yang Xiang,Ziqian Xie,Cui Tao,Degui Zhi

DOI: https://doi.org/10.1038/s41746-021-00455-y

IF: 15.2

2021-05-20

npj Digital Medicine

Abstract:Abstract Deep learning (DL)-based predictive models from electronic health records (EHRs) deliver impressive performance in many clinical tasks. Large training cohorts, however, are often required by these models to achieve high accuracy, hindering the adoption of DL-based models in scenarios with limited training data. Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous successes in the natural language processing domain. The pretraining of BERT on a very large training corpus generates contextualized embeddings that can boost the performance of models trained on smaller datasets. Inspired by BERT, we propose Med-BERT, which adapts the BERT framework originally developed for the text domain to the structured EHR domain. Med-BERT is a contextualized embedding model pretrained on a structured EHR dataset of 28,490,650 patients. Fine-tuning experiments showed that Med-BERT substantially improves the prediction accuracy, boosting the area under the receiver operating characteristics curve (AUC) by 1.21–6.14% in two disease prediction tasks from two clinical databases. In particular, pretrained Med-BERT obtains promising performances on tasks with small fine-tuning training sets and can boost the AUC by more than 20% or obtain an AUC as high as a model trained on a training set ten times larger, compared with deep learning models without Med-BERT. We believe that Med-BERT will benefit disease prediction studies with small local training datasets, reduce data collection expenses, and accelerate the pace of artificial intelligence aided healthcare.

health care sciences & services,medical informatics
Modified Bidirectional Encoder Representations From Transformers Extractive Summarization Model for Hospital Information Systems Based on Character-Level Tokens (AlphaBERT): Development and Performance Evaluation

Yen-Pin Chen,Yi-Ying Chen,Jr-Jiun Lin,Chien-Hua Huang,Feipei Lai

DOI: https://doi.org/10.2196/17787

2020-04-29

Abstract:Background: Doctors must care for many patients simultaneously, and it is time-consuming to find and examine all patients' medical histories. Discharge diagnoses provide hospital staff with sufficient information to enable handling multiple patients; however, the excessive amount of words in the diagnostic sentences poses problems. Deep learning may be an effective solution to overcome this problem, but the use of such a heavy model may also add another obstacle to systems with limited computing resources. Objective: We aimed to build a diagnoses-extractive summarization model for hospital information systems and provide a service that can be operated even with limited computing resources. Methods: We used a Bidirectional Encoder Representations from Transformers (BERT)-based structure with a two-stage training method based on 258,050 discharge diagnoses obtained from the National Taiwan University Hospital Integrated Medical Database, and the highlighted extractive summaries written by experienced doctors were labeled. The model size was reduced using a character-level token, the number of parameters was decreased from 108,523,714 to 963,496, and the model was pretrained using random mask characters in the discharge diagnoses and International Statistical Classification of Diseases and Related Health Problems sets. We then fine-tuned the model using summary labels and cleaned up the prediction results by averaging all probabilities for entire words to prevent character level-induced fragment words. Model performance was evaluated against existing models BERT, BioBERT, and Long Short-Term Memory (LSTM) using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) L score, and a questionnaire website was built to collect feedback from more doctors for each summary proposal. Results: The area under the receiver operating characteristic curve values of the summary proposals were 0.928, 0.941, 0.899, and 0.947 for BERT, BioBERT, LSTM, and the proposed model (AlphaBERT), respectively. The ROUGE-L scores were 0.697, 0.711, 0.648, and 0.693 for BERT, BioBERT, LSTM, and AlphaBERT, respectively. The mean (SD) critique scores from doctors were 2.232 (0.832), 2.134 (0.877), 2.207 (0.844), 1.927 (0.910), and 2.126 (0.874) for reference-by-doctor labels, BERT, BioBERT, LSTM, and AlphaBERT, respectively. Based on the paired t test, there was a statistically significant difference in LSTM compared to the reference (P<.001), BERT (P=.001), BioBERT (P<.001), and AlphaBERT (P=.002), but not in the other models. Conclusions: Use of character-level tokens in a BERT model can greatly decrease the model size without significantly reducing performance for diagnoses summarization. A well-developed deep-learning model will enhance doctors' abilities to manage patients and promote medical studies by providing the capability to use extensive unstructured free-text notes.

[Flourescent amoebic antibody test in a patient with secondary carcinoma of the liver (author's transl)].

Evaluating robustness of language models for chief complaint extraction from patient-generated text

Using a clinical narrative-aware pre-trained language model for predicting emergency department patient disposition and unscheduled return visits

Manganese Reduction by Microbes from Oxic Regions of the Lake Vanda (Antarctica) Water Column

Autocompletion of Chief Complaints in the Electronic Health Records using Large Language Models

Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing

Understanding patient complaint characteristics using contextual clinical BERT embeddings

[Drug permeation through artificial lipoid membranes. 19. The mechanism of ion pair permeation--alkali ion countertransport].

Incorporating Domain Knowledge Into Language Models by Using Graph Convolutional Networks for Assessing Semantic Textual Similarity: Model Development and Performance Comparison

Multi-Label Classification in Patient-Doctor Dialogues With the RoBERTa-WWM-ext + CNN (Robustly Optimized Bidirectional Encoder Representations From Transformers Pretraining Approach With Whole Word Masking Extended Combining a Convolutional Neural Network) Model: Named Entity Study

Enhancing efficiency and capacity of telehealth services with intelligent triage: a bidirectional LSTM neural network model employing character embedding

Regles positionnelles iteratives, principe majoritaire et preferences unimodales

Emergency Department Decision Support using Clinical Pseudo-notes

Leveraging Pre-trained and Transformer-derived Embeddings from EHRs to Characterize Heterogeneity Across Alzheimer's Disease and Related Dementias

Emergency department disposition prediction using a deep neural network with integrated clinical narratives and structured data

Clinical Named Entity Recognition using Contextualized Token Representations

CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks

A pre-trained language model for emergency department intervention prediction using routine physiological data and clinical narratives

Natural products containing a nitrogen-nitrogen bond.

Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction

Modified Bidirectional Encoder Representations From Transformers Extractive Summarization Model for Hospital Information Systems Based on Character-Level Tokens (AlphaBERT): Development and Performance Evaluation