Watch Where and How You Stick Pins When Playing With Voodoo Correlations

C. McCormick

DOI: https://doi.org/10.1080/00221309.2012.737872

2013-01-01

The Journal of General Psychology

Abstract:ABSTRACT The author comments on the (mis?)portrayal of her research in an article by Brand and Bradley (2012).

What problem does this paper attempt to address?

ESR studies on membrane fluidity of Chinese hamster ovary cells grown on microcarriers and in suspension.

C. Lai,L. Hopwood,H. Swartz

DOI: https://doi.org/10.1016/0014-4827(80)90022-1

IF: 4.145

1980-12-01

Experimental Cell Research

Abstract:
Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

Emre Sezgin,Syed-Amad Hussain,Steve Rust,Yungui Huang

DOI: https://doi.org/10.2196/43014

2023-03-07

JMIR Formative Research

Abstract:Background Patient-generated health data (PGHD) captured via smart devices or digital health technologies can reflect an individual health journey. PGHD enables tracking and monitoring of personal health conditions, symptoms, and medications out of the clinic, which is crucial for self-care and shared clinical decisions. In addition to self-reported measures and structured PGHD (eg, self-screening, sensor-based biometric data), free-text and unstructured PGHD (eg, patient care note, medical diary) can provide a broader view of a patient’s journey and health condition. Natural language processing (NLP) is used to process and analyze unstructured data to create meaningful summaries and insights, showing promise to improve the utilization of PGHD. Objective Our aim is to understand and demonstrate the feasibility of an NLP pipeline to extract medication and symptom information from real-world patient and caregiver data. Methods We report a secondary data analysis, using a data set collected from 24 parents of children with special health care needs (CSHCN) who were recruited via a nonrandom sampling approach. Participants used a voice-interactive app for 2 weeks, generating free-text patient notes (audio transcription or text entry). We built an NLP pipeline using a zero-shot approach (adaptive to low-resource settings). We used named entity recognition (NER) and medical ontologies (RXNorm and SNOMED CT [Systematized Nomenclature of Medicine Clinical Terms]) to identify medication and symptoms. Sentence-level dependency parse trees and part-of-speech tags were used to extract additional entity information using the syntactic properties of a note. We assessed the data; evaluated the pipeline with the patient notes; and reported the precision, recall, and F1 scores. Results In total, 87 patient notes are included (audio transcriptions n=78 and text entries n=9) from 24 parents who have at least one CSHCN. The participants were between the ages of 26 and 59 years. The majority were White (n=22, 92%), had more than one child (n=16, 67%), lived in Ohio (n=22, 92%), had mid- or upper-mid household income (n=15, 62.5%), and had higher level education (n=24, 58%). Out of 87 notes, 30 were drug and medication related, and 46 were symptom related. We captured medication instances (medication, unit, quantity, and date) and symptoms satisfactorily (precision >0.65, recall >0.77, F1>0.72). These results indicate the potential when using NER and dependency parsing through an NLP pipeline on information extraction from unstructured PGHD. Conclusions The proposed NLP pipeline was found to be feasible for use with real-world unstructured PGHD to accomplish medication and symptom extraction. Unstructured PGHD can be leveraged to inform clinical decision-making, remote monitoring, and self-care including medical adherence and chronic disease management. With customizable information extraction methods using NER and medical ontologies, NLP models can feasibly extract a broad range of clinical information from unstructured PGHD in low-resource settings (eg, a limited number of patient notes or training data).
Application of natural language processing to identify social needs from patient medical notes: development and assessment of a scalable, performant, and rule-based model in an integrated healthcare delivery system

Geoffrey M Gray,Ayah Zirikly,Luis M Ahumada,Masoud Rouhizadeh,Thomas Richards,Christopher Kitchen,Iman Foroughmand,Elham Hatef

DOI: https://doi.org/10.1093/jamiaopen/ooad085

2023-10-04

JAMIA Open

Abstract:Objectives: To develop and test a scalable, performant, and rule-based model for identifying 3 major domains of social needs (residential instability, food insecurity, and transportation issues) from the unstructured data in electronic health records (EHRs). Materials and methods: We included patients aged 18 years or older who received care at the Johns Hopkins Health System (JHHS) between July 2016 and June 2021 and had at least 1 unstructured (free-text) note in their EHR during the study period. We used a combination of manual lexicon curation and semiautomated lexicon creation for feature development. We developed an initial rules-based pipeline (Match Pipeline) using 2 keyword sets for each social needs domain. We performed rule-based keyword matching for distinct lexicons and tested the algorithm using an annotated dataset comprising 192 patients. Starting with a set of expert-identified keywords, we tested the adjustments by evaluating false positives and negatives identified in the labeled dataset. We assessed the performance of the algorithm using measures of precision, recall, and F1 score. Results: The algorithm for identifying residential instability had the best overall performance, with a weighted average for precision, recall, and F1 score of 0.92, 0.84, and 0.92 for identifying patients with homelessness and 0.84, 0.82, and 0.79 for identifying patients with housing insecurity. Metrics for the food insecurity algorithm were high but the transportation issues algorithm was the lowest overall performing metric. Discussion: The NLP algorithm in identifying social needs at JHHS performed relatively well and would provide the opportunity for implementation in a healthcare system. Conclusion: The NLP approach developed in this project could be adapted and potentially operationalized in the routine data processes of a healthcare system.
[Cholestasis. Diagnostic value of immunoelectrophoresis].

J. Balanzó,G. Buendía,A. Ferragut,R. Vidal-Ribas

1969-10-15

Abstract:
Practical use case of natural language processing for observational clinical research data retrieval from electronic health records: AssistMED project

Cezary Maciejewski,Krzysztof Ozierański,Mikołaj Basza,Adam Barwiołek,Michalina Ciurla,Aleksandra Bożym,Maciej J Krajsman,Piotr Lodziński,Grzegorz Opolski,Marcin Grabowski,Andrzej Cacko,Paweł Balsam,Maciej J. Krajsman

DOI: https://doi.org/10.20452/pamw.16704

2024-03-19

Polish Archives of Internal Medicine

Abstract:INTRODUCTION: Electronic health records (EHR) contain data valuable for clinical research but in textual format, requiring encoding to databases by a human- a lengthy and costly process. Natural language processing (NLP) is a computational technique that allows text analysis.OBJECTIVES: To demonstrate a practical use case of NLP for a large retrospective study cohort characterization and compare it to a human retrieval.PATIENTS AND METHODS: Anonymized discharge documentation of 10314 patients from the cardiology tertiary care department was analyzed for inclusion in the CRAFT registry (NCT02987062) of patients with atrial fibrillation (AF). Extensive clinical characteristics regarding concomitant diseases, medications, daily dosage and echocardiography were collected manually and through NLP.RESULTS: There were 3030 and 3029 patients identified by human and NLP-based approaches, respectively, reflecting 99.93% accuracy of NLP in detecting AF. Comprehensive baseline patient characteristics by NLP was faster than human analysis (3 hours and 15 minutes vs 71 hours and 12 minutes). The calculated CHA2DS2VASc and HAS-BLED scores based on both methods did not differ (human vs NLP; median, IQR, P value): 3 (2-5) vs 3 (2-5) P = 0.74 and 1 (1-2) vs 1 (1-2) P = 0.63. For most data, an almost perfect agreement between NLP and human retrieved characteristics was found; daily dosage identification was the least accurate NLP feature. Similar conclusions on cohort characteristics would be made; however, daily dosage detection for some drug groups would require additional human validation in the NLP-based cohort.CONCLUSIONS: NLP utilization on EHR may accelerate acquisition and provide accurate data for a retrospective study.

medicine, general & internal
Identification of suicidal behavior among psychiatrically hospitalized adolescents using natural language processing and machine learning of electronic health records

Nicholas J. Carson,Brian Mullin,Maria Jose Sanchez,Frederick Lu,Kelly Yang,Michelle Menezes,Benjamin Lê Cook

DOI: https://doi.org/10.1371/journal.pone.0211116

IF: 3.7

2019-02-19

PLoS ONE

Abstract:OBJECTIVE: The rapid proliferation of machine learning research using electronic health records to classify healthcare outcomes offers an opportunity to address the pressing public health problem of adolescent suicidal behavior. We describe the development and evaluation of a machine learning algorithm using natural language processing of electronic health records to identify suicidal behavior among psychiatrically hospitalized adolescents.METHODS: Adolescents hospitalized on a psychiatric inpatient unit in a community health system in the northeastern United States were surveyed for history of suicide attempt in the past 12 months. A total of 73 respondents had electronic health records available prior to the index psychiatric admission. Unstructured clinical notes were downloaded from the year preceding the index inpatient admission. Natural language processing identified phrases from the notes associated with the suicide attempt outcome. We enriched this group of phrases with a clinically focused list of terms representing known risk and protective factors for suicide attempt in adolescents. We then applied the random forest machine learning algorithm to develop a classification model. The model performance was evaluated using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy.RESULTS: The final model had a sensitivity of 0.83, specificity of 0.22, AUC of 0.68, a PPV of 0.42, NPV of 0.67, and an accuracy of 0.47. The terms mostly highly associated with suicide attempt clustered around terms related to suicide, family members, psychiatric disorders, and psychotropic medications.CONCLUSION: This analysis demonstrates modest success of a natural language processing and machine learning approach to identifying suicide attempt among a small sample of hospitalized adolescents in a psychiatric setting.

multidisciplinary sciences
Can natural language processing models extract and classify instances of interpersonal violence in mental healthcare electronic records: an applied evaluative study

Riley Botelle,Vishal Bhavsar,Giouliana Kadra-Scalzo,Aurelie Mascio,Marcus V Williams,Angus Roberts,Sumithra Velupillai,Robert Stewart

DOI: https://doi.org/10.1136/bmjopen-2021-052911

IF: 3.006

2022-02-16

BMJ Open

Abstract:Objective: This paper evaluates the application of a natural language processing (NLP) model for extracting clinical text referring to interpersonal violence using electronic health records (EHRs) from a large mental healthcare provider. Design: A multidisciplinary team iteratively developed guidelines for annotating clinical text referring to violence. Keywords were used to generate a dataset which was annotated (ie, classified as affirmed, negated or irrelevant) for: presence of violence, patient status (ie, as perpetrator, witness and/or victim of violence) and violence type (domestic, physical and/or sexual). An NLP approach using a pretrained transformer model, BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) was fine-tuned on the annotated dataset and evaluated using 10-fold cross-validation. Setting: We used the Clinical Records Interactive Search (CRIS) database, comprising over 500 000 de-identified EHRs of patients within the South London and Maudsley NHS Foundation Trust, a specialist mental healthcare provider serving an urban catchment area. Participants: Searches of CRIS were carried out based on 17 predefined keywords. Randomly selected text fragments were taken from the results for each keyword, amounting to 3771 text fragments from the records of 2832 patients. Outcome measures: We estimated precision, recall and F1 score for each NLP model. We examined sociodemographic and clinical variables in patients giving rise to the text data, and frequencies for each annotated violence characteristic. Results: Binary classification models were developed for six labels (violence presence, perpetrator, victim, domestic, physical and sexual). Among annotations affirmed for the presence of any violence, 78% (1724) referred to physical violence, 61% (1350) referred to patients as perpetrator and 33% (731) to domestic violence. NLP models' precision ranged from 89% (perpetrator) to 98% (sexual); recall ranged from 89% (victim, perpetrator) to 97% (sexual). Conclusions: State of the art NLP models can extract and classify clinical text on violence from EHRs at acceptable levels of scale, efficiency and accuracy.
Learning for Automotive Collision Avoidance and Autonomous Control

D. Pomerleau

DOI: https://doi.org/10.1016/b978-1-55860-377-6.50080-3

1995-07-09

Abstract:
Generation and evaluation of artificial mental health records for Natural Language Processing

Julia Ive,Natalia Viani,Joyce Kam,Lucia Yin,Somain Verma,Stephen Puntis,Rudolf N. Cardinal,Angus Roberts,Robert Stewart,Sumithra Velupillai

DOI: https://doi.org/10.1038/s41746-020-0267-x

IF: 15.2

2020-05-14

npj Digital Medicine

Abstract:Abstract A serious obstacle to the development of Natural Language Processing (NLP) methods in the clinical domain is the accessibility of textual data. The mental health domain is particularly challenging, partly because clinical documentation relies heavily on free text that is difficult to de-identify completely. This problem could be tackled by using artificial medical data. In this work, we present an approach to generate artificial clinical documents. We apply this approach to discharge summaries from a large mental healthcare provider and discharge summaries from an intensive care unit. We perform an extensive intrinsic evaluation where we (1) apply several measures of text preservation; (2) measure how much the model memorises training data; and (3) estimate clinical validity of the generated text based on a human evaluation task. Furthermore, we perform an extrinsic evaluation by studying the impact of using artificial text in a downstream NLP text classification task. We found that using this artificial data as training data can lead to classification results that are comparable to the original results. Additionally, using only a small amount of information from the original data to condition the generation of the artificial data is successful, which holds promise for reducing the risk of these artificial data retaining rare information from the original data. This is an important finding for our long-term goal of being able to generate artificial clinical data that can be released to the wider research community and accelerate advances in developing computational methods that use healthcare data.

health care sciences & services,medical informatics
Improving ascertainment of suicidal ideation and suicide attempt with natural language processing

Cosmin A. Bejan,Michael Ripperger,Drew Wilimitis,Ryan Ahmed,JooEun Kang,Katelyn Robinson,Theodore J. Morley,Douglas M. Ruderfer,Colin G. Walsh

DOI: https://doi.org/10.1038/s41598-022-19358-3

IF: 4.6

2022-09-09

Scientific Reports

Abstract:Methods relying on diagnostic codes to identify suicidal ideation and suicide attempt in Electronic Health Records (EHRs) at scale are suboptimal because suicide-related outcomes are heavily under-coded. We propose to improve the ascertainment of suicidal outcomes using natural language processing (NLP). We developed information retrieval methodologies to search over 200 million notes from the Vanderbilt EHR. Suicide query terms were extracted using word2vec. A weakly supervised approach was designed to label cases of suicidal outcomes. The NLP validation of the top 200 retrieved patients showed high performance for suicidal ideation (area under the receiver operator curve [AUROC]: 98.6, 95% confidence interval [CI] 97.1–99.5) and suicide attempt (AUROC: 97.3, 95% CI 95.2–98.7). Case extraction produced the best performance when combining NLP and diagnostic codes and when accounting for negated suicide expressions in notes. Overall, we demonstrated that scalable and accurate NLP methods can be developed to identify suicidal behavior in EHRs to enhance prevention efforts, predictive models, and precision medicine.

multidisciplinary sciences
Enhancing Patient Experience by Automating and Transforming Free Text into Actionable Consumer Insights: A Natural Language Processing (NLP) Approach

Ela Vashishtha,Himanshu Kapoor

DOI: https://doi.org/10.52403/ijhsr.20231038

2023-10-26

International Journal of Health Sciences and Research

Abstract:Background: The digitalization of healthcare has expanded patient access to care delivery through various virtual platforms. These platforms accumulate a wealth of patient feedback data, offering valuable insights into care perception and experience. Natural Language Processing (NLP) provides a means to efficiently analyze this unstructured patient feedback, enabling healthcare organizations to enhance care quality and patient experience. Methods: This study collected 19,000 comments from diverse sources, including surveys and social media, spanning from January 1, 2018, to June 30, 2021. The data underwent preprocessing, including lowercase conversion, special character removal, and lemmatization. A "hot word" list was employed to identify critical patient safety concerns. NLP models, including sentiment analysis and text classification, were used to analyze patient feedback. Results: The sentiment analysis revealed that most patient comments were positive or neutral, with 34.5% positive, 52.6% negative, and 12.9% neutral sentiments. Aggregate analysis identified access and login issues as a primary source of dissatisfaction, with 78% of related comments expressing negativity. The study also utilized Word2Vec to uncover word associations, highlighting positive associations with "staff" and negative associations with "website" and "insurance." Hot word detection identified 580 comments requiring immediate attention for patient safety. Conclusions: NLP-enabled analysis of patient feedback offers actionable insights for healthcare organizations. It identifies areas for improvement, tracks sentiment trends, and aids in patient engagement. Additionally, NLP benefits research and development by accelerating drug discovery and improving healthcare outcomes. Despite limitations, NLP's role in healthcare data analysis is poised to grow, benefiting both providers and consumers. This study recommends proactive use of NLP tools to enhance care delivery and patient experience. Key words: Natural Language Processing, Context-aware Algorithms, Quality Improvement Framework, Text Classification, Electronic Health Records, Machine Learning, Word2Vec, Sentiment Analysis, Word2Vec, Hot Word List, Frequency Distribution, Patient Feedback.
Scalable information extraction from free text electronic health records using large language models

Bowen Gu,Vivian Shao,Ziqian Liao,Valentina Carducci,Santiago Romero-Brufau,Jie Yang,Rishi Desai

DOI: https://doi.org/10.1101/2024.08.08.24311237

2024-08-10

Abstract:Background: A vast amount of potentially useful information such as description of patient symptoms, family, and social history is recorded as free-text notes in electronic health records (EHRs) but is difficult to reliably extract at scale, limiting their utility in research. This study aims to assess whether an "out of the box" implementation of open-source large language models (LLMs) without any fine-tuning can accurately extract social determinants of health (SDoH) data from free-text clinical notes. Methods: We conducted a cross-sectional study using EHR data from the Mass General Brigham (MGB) system, analyzing free-text notes for SDoH information. We selected a random sample of 200 patients and manually labeled nine SDoH aspects. Eight advanced open-source LLMs were evaluated against a baseline pattern-matching model. Two human reviewers provided the manual labels, achieving 93% inter-annotator agreement. LLM performance was assessed using accuracy metrics for overall, mentioned, and non-mentioned SDoH, and macro F1 scores. Results: LLMs outperformed the baseline pattern-matching approach, particularly for explicitly mentioned SDoH, achieving up to 40% higher Accuracy_mentioned. openchat_3.5 was the best performing model, surpassing the baseline in overall accuracy across all nine SDoH aspects. The refined pipeline with prompt engineering reduced hallucinations and improved accuracy. Conclusions: Open-source LLMs are effective and scalable tools for extracting SDoH from unstructured EHRs, surpassing traditional pattern-matching methods. Further refinement and domain-specific training could enhance their utility in clinical research and predictive analytics, improving healthcare outcomes and addressing health disparities

Health Informatics
C-reactive protein: an early critical sign of clozapine-related myocarditis

S. Fehily,Rosaria Forlano,P. Fitzgerald

DOI: https://doi.org/10.1177/1039856215604482

2016-04-01

Australasian Psychiatry

Abstract:Objective: We present a case of clozapine-related myocarditis, with a rising C-reactive protein as the only initial evidence supporting the diagnosis. Method: An otherwise healthy young male presenting with treatment-resistant schizophrenia was started on clozapine. Monitoring was performed. Results: At day 18 he developed fever, tachycardia and a raised C-reactive protein, while troponin levels and echocardiogram remained normal. Conclusions: Current protocols monitoring for myocarditis have their limitations and can often only be used to support a presumptive diagnosis of myocarditis. In keeping with the current clozapine monitoring guidelines, we demonstrate that a rise in C-reactive protein levels can be a critical early sign of myocarditis warranting close monitoring and serious consideration for cessation of clozapine.
Cohort Design and Natural Language Processing to Reduce Bias in Electronic Health Records Research: The Community Care Cohort Project

Shaan Khurshid,Christopher Reeder,Lia X. Harrington,Pulkit Singh,Gopal Sarma,Samuel F. Friedman,Paolo Di Achille,Nathaniel Diamant,Jonathan W. Cunningham,Ashby C. Turner,Emily S. Lau,Julian S. Haimovich,Mostafa A. Al-Alusi,Xin Wang,Marcus D.R. Klarqvist,Jeffrey M. Ashburner,Christian Diedrich,Mercedeh Ghadessi,Johanna Mielke,Hanna M. Eilken,Alice McElhinney,Andrea Derix,Steven J. Atlas,Patrick T. Ellinor,Anthony A. Philippakis,Christopher D. Anderson,Jennifer E. Ho,Puneet Batra,Steven A. Lubitz

DOI: https://doi.org/10.1101/2021.05.26.21257872

2021-05-30

Abstract:ABSTRACT Background Electronic health records (EHRs) promise to enable broad-ranging discovery with power exceeding that of conventional research cohort studies. However, research using EHR datasets may be subject to selection bias, which can be compounded by missing data, limiting the generalizability of derived insights. Methods Mass General Brigham (MGB) is a large New England-based healthcare network comprising seven tertiary care and community hospitals with associated outpatient practices. Within an MGB-based EHR warehouse of >3.5 million individuals with at least one ambulatory care visit, we approximated a community-based cohort study by selectively sampling individuals longitudinally attending primary care practices between 2001-2018 (n=520,868), which we named the Community Care Cohort Project (C3PO). We also utilized pre-trained deep natural language processing (NLP) models to recover vital signs (i.e., height, weight, and blood pressure) from unstructured notes in the EHR. We assessed the validity of C3PO by deploying established risk models including the Pooled Cohort Equations (PCE) and the Cohorts for Aging and Genomic Epidemiology Atrial Fibrillation (CHARGE-AF) score, and compared model performance in C3PO to that observed within typical EHR Convenience Samples which included all individuals from the same parent EHR with sufficient data to calculate each score but without a requirement for longitudinal primary care. All analyses were facilitated by the JEDI Extractive Data Infrastructure pipeline which we designed to efficiently aggregate EHR data within a unified framework conducive to regular updates. Results C3PO includes 520,868 individuals (mean age 48 years, 61% women, median follow-up 7.2 years, median primary care visits per individual 13). Estimated using reports, C3PO contains over 2.9 million electrocardiograms, 450,000 echocardiograms, 12,000 cardiac magnetic resonance images, and 75 million narrative notes. Using tabular data alone, 286,009 individuals (54.9%) had all vital signs available at baseline, which increased to 358,411 (68.8%) after NLP recovery (31% reduction in missingness). Among individuals with both NLP and tabular data available, NLP-extracted and tabular vital signs obtained on the same day were highly correlated (e.g., Pearson r range 0.95-0.99, p<0.01 for all). Both the PCE models (c-index range 0.724-0.770) and CHARGE-AF (c-index 0.782, 95% 0.777-0.787) demonstrated good discrimination. As compared to the Convenience Samples, AF and MI/stroke incidence rates in C3PO were lower and calibration error was smaller for both PCE (integrated calibration index range 0.012-0.030 vs. 0.028-0.046) and CHARGE-AF (0.028 vs. 0.036). Conclusions Intentional sampling of individuals receiving regular ambulatory care and use of NLP to recover missing data have the potential to reduce bias in EHR research and maximize generalizability of insights.
Automated recognition of functioning, activity and participation in COVID-19 from electronic patient records by natural language processing: a proof- of- concept

Carel G. M. Meskers,Sabina van der Veen,Jenia Kim,Caroline J. W. Meskers,Quirine T. S. Smit,Stella Verkijk,Edwin Geleijn,Guy A. M. Widdershoven,Piek T. J. M. Vossen,Marike van der Leeden

DOI: https://doi.org/10.1080/07853890.2021.2025418

IF: 5.348

2022-01-18

Annals of Medicine

Abstract:PURPOSE: To address the feasibility, reliability and internal validity of natural language processing (NLP) for automated functional assessment of hospitalised COVID-19 patients in key International Classification of Functioning, Disability and Health (ICF) categories and levels from unstructured text in electronic health records (EHR) from a large teaching hospital.MATERIALS AND METHODS: Eight human annotators assigned four ICF categories to relevant sentences: Emotional functions, Exercise tolerance, Walking and Moving, Work and Employment and their ICF levels (Functional Ambulation Categories for Walking and Moving, metabolic equivalents for Exercise tolerance). A linguistic neural network-based model was trained on 80% of the annotated sentences; inter-annotator agreement (IAA, Cohen's kappa), a weighted score of precision and recall (F1) and RMSE for level detection were assessed for the remaining 20%.RESULTS: In total 4112 sentences of non-COVID-19 and 1061 of COVID-19 patients were annotated. Average IAA was 0.81; F1 scores were 0.7 for Walking and Moving and Emotional functions; RMSE for Walking and Moving (5- level scale) was 1.17 for COVID-19 patients.CONCLUSION: Using a limited amount of annotated EHR sentences, a proof-of-concept was obtained for automated functional assessment of COVID-19 patients in ICF categories and levels. This allows for instantaneous assessment of the functional consequences of new diseases like COVID-19 for large numbers of patients.Key messagesHospitalised Covid-19 survivors may persistently suffer from low physical and mental functioning and a reduction in overall quality of life requiring appropriate and personalised rehabilitation strategies.For this, assessment of functioning within multiple domains and categories of the International Classification of Function is required, which is cumbersome using structured data.We show a proof-of-concept using Natural Language Processing techniques to automatically derive the aforementioned information from free-text notes within the Electronic Health Record of a large academic teaching hospital.

medicine, general & internal
Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults

Majid Afshar,Sabrina Adelaine,Felice Resnik,Marlon P Mundt,John Long,Margaret Leaf,Theodore Ampian,Graham J Wills,Benjamin Schnapp,Michael Chao,Randy Brown,Cara Joyce,Brihat Sharma,Dmitriy Dligach,Elizabeth S Burnside,Jane Mahoney,Matthew M Churpek,Brian W Patterson,Frank Liao

DOI: https://doi.org/10.2196/44977

2023-04-20

Abstract:Background: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. Objective: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. Methods: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. Results: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. Conclusions: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence-driven CDS. Trial registration: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480.
Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing

Sifei Han,Robert F. Zhang,Lingyun Shi,Russell Richie,Haixia Liu,Andrew Tseng,Wei Quan,Neal Ryan,David Brent,Fuchiang R. Tsui

DOI: https://doi.org/10.1016/j.jbi.2021.103984

IF: 8

2022-03-01

Journal of Biomedical Informatics

Abstract:OBJECTIVE: Social determinants of health (SDOH) are non-medical factors that can profoundly impact patient health outcomes. However, SDOH are rarely available in structured electronic health record (EHR) data such as diagnosis codes, and more commonly found in unstructured narrative clinical notes. Hence, identifying social context from unstructured EHR data has become increasingly important. Yet, previous work on using natural language processing to automate extraction of SDOH from text (a) usually focuses on an ad hoc selection of SDOH, and (b) does not use the latest advances in deep learning. Our objective was to advance automatic extraction of SDOH from clinical text by (a) systematically creating a set of SDOH based on standard biomedical and psychiatric ontologies, and (b) training state-of-the-art deep neural networks to extract mentions of these SDOH from clinical notes.DESIGN: A retrospective cohort study.SETTING AND PARTICIPANTS: Data were extracted from the Medical Information Mart for Intensive Care (MIMIC-III) database. The corpus comprised 3,504 social related sentences from 2,670 clinical notes.METHODS: We developed a framework for automated classification of multiple SDOH categories. Our dataset comprised narrative clinical notes under the "Social Work" category in the MIMIC-III Clinical Database. Using standard terminologies, SNOMED-CT and DSM-IV, we systematically curated a set of 13 SDOH categories and created annotation guidelines for these. After manually annotating the 3,504 sentences, we developed and tested three deep neural network (DNN) architectures - convolutional neural network (CNN), long short-term memory (LSTM) network, and the Bidirectional Encoder Representations from Transformers (BERT) - for automated detection of eight SDOH categories. We also compared these DNNs to three baselines models: (1) cTAKES, as well as (2) L2-regularized logistic regression and (3) random forests on bags-of-words. Model evaluation metrics included micro- and macro- F1, and area under the receiver operating characteristic curve (AUC).RESULTS: All three DNN models accurately classified all SDOH categories (minimum micro-F1 = 0.632, minimum macro-AUC = 0.854). Compared to the CNN and LSTM, BERT performed best in most key metrics (micro-F1 = 0.690, macro-AUC = 0.907). The BERT model most effectively identified the "occupational" category (F1 = 0.774, AUC = 0.965) and least effectively identified the "non-SDOH" category (F = 0.491, AUC = 0.788). BERT outperformed cTAKES in distinguishing social vs non-social sentences (BERT F1 = 0.87 vs. cTAKES F1 = 0.06), and outperformed logistic regression (micro-F1 = 0.649, macro-AUC = 0.696) and random forest (micro-F1 = 0.502, macro-AUC = 0.523) trained on bag-of-words.CONCLUSIONS: Our study framework with DNN models demonstrated improved performance for efficiently identifying a systematic range of SDOH categories from clinical notes in the EHR. Improved identification of patient SDOH may further improve healthcare outcomes.

medical informatics,computer science, interdisciplinary applications
Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review

Mustafa Khanbhai,Patrick Anyadi,Joshua Symons,Kelsey Flott,Ara Darzi,Erik Mayer

DOI: https://doi.org/10.1136/bmjhci-2020-100262

2021-03-01

BMJ Health and Care Informatics

Abstract:Objectives Unstructured free-text patient feedback contains rich information, and analysing these data manually would require a lot of personnel resources which are not available in most healthcare organisations.To undertake a systematic review of the literature on the use of natural language processing (NLP) and machine learning (ML) to process and analyse free-text patient experience data. Methods Databases were systematically searched to identify articles published between January 2000 and December 2019 examining NLP to analyse free-text patient feedback. Due to the heterogeneous nature of the studies, a narrative synthesis was deemed most appropriate. Data related to the study purpose, corpus, methodology, performance metrics and indicators of quality were recorded. Results Nineteen articles were included. The majority (80%) of studies applied language analysis techniques on patient feedback from social media sites (unsolicited) followed by structured surveys (solicited). Supervised learning was frequently used (n=9), followed by unsupervised (n=6) and semisupervised (n=3). Comments extracted from social media were analysed using an unsupervised approach, and free-text comments held within structured surveys were analysed using a supervised approach. Reported performance metrics included the precision, recall and F-measure, with support vector machine and Naïve Bayes being the best performing ML classifiers. Conclusion NLP and ML have emerged as an important tool for processing unstructured free text. Both supervised and unsupervised approaches have their role depending on the data source. With the advancement of data analysis tools, these techniques may be useful to healthcare organisations to generate insight from the volumes of unstructured free-text data.
Extracting social determinants of health from electronic health records using natural language processing: a systematic review

Braja G Patra,Mohit M Sharma,Veer Vekaria,Prakash Adekkanattu,Olga V Patterson,Benjamin Glicksberg,Lauren A Lepow,Euijung Ryu,Joanna M Biernacka,Al’ona Furmanchuk,Thomas J George,William Hogan,Yonghui Wu,Xi Yang,Jiang Bian,Myrna Weissman,Priya Wickramaratne,J John Mann,Mark Olfson,Thomas R Campion,Mark Weiner,Jyotishman Pathak

DOI: https://doi.org/10.1093/jamia/ocab170

2021-10-06

Journal of the American Medical Informatics Association

Abstract:Abstract Objective Social determinants of health (SDoH) are nonclinical dispositions that impact patient health risks and clinical outcomes. Leveraging SDoH in clinical decision-making can potentially improve diagnosis, treatment planning, and patient outcomes. Despite increased interest in capturing SDoH in electronic health records (EHRs), such information is typically locked in unstructured clinical notes. Natural language processing (NLP) is the key technology to extract SDoH information from clinical text and expand its utility in patient care and research. This article presents a systematic review of the state-of-the-art NLP approaches and tools that focus on identifying and extracting SDoH data from unstructured clinical text in EHRs. Materials and Methods A broad literature search was conducted in February 2021 using 3 scholarly databases (ACL Anthology, PubMed, and Scopus) following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A total of 6402 publications were initially identified, and after applying the study inclusion criteria, 82 publications were selected for the final review. Results Smoking status (n = 27), substance use (n = 21), homelessness (n = 20), and alcohol use (n = 15) are the most frequently studied SDoH categories. Homelessness (n = 7) and other less-studied SDoH (eg, education, financial problems, social isolation and support, family problems) are mostly identified using rule-based approaches. In contrast, machine learning approaches are popular for identifying smoking status (n = 13), substance use (n = 9), and alcohol use (n = 9). Conclusion NLP offers significant potential to extract SDoH data from narrative clinical notes, which in turn can aid in the development of screening tools, risk prediction models, and clinical decision support systems.

information science & library science,computer science, information systems, interdisciplinary applications,health care sciences & services,medical informatics
Using Social Media to Help Understand Patient-Reported Health Outcomes of Post-COVID-19 Condition: Natural Language Processing Approach

Elham Dolatabadi,Diana Moyano,Michael Bales,Sofija Spasojevic,Rohan Bhambhoria,Junaid Bhatti,Shyamolima Debnath,Nicholas Hoell,Xin Li,Celine Leng,Sasha Nanda,Jad Saab,Esmat Sahak,Fanny Sie,Sara Uppal,Nirma Khatri Vadlamudi,Antoaneta Vladimirova,Artur Yakimovich,Xiaoxue Yang,Sedef Akinli Kocak,Angela M Cheung

DOI: https://doi.org/10.2196/45767

2023-09-19

Abstract:Background: While scientific knowledge of post-COVID-19 condition (PCC) is growing, there remains significant uncertainty in the definition of the disease, its expected clinical course, and its impact on daily functioning. Social media platforms can generate valuable insights into patient-reported health outcomes as the content is produced at high resolution by patients and caregivers, representing experiences that may be unavailable to most clinicians. Objective: In this study, we aimed to determine the validity and effectiveness of advanced natural language processing approaches built to derive insight into PCC-related patient-reported health outcomes from social media platforms Twitter and Reddit. We extracted PCC-related terms, including symptoms and conditions, and measured their occurrence frequency. We compared the outputs with human annotations and clinical outcomes and tracked symptom and condition term occurrences over time and locations to explore the pipeline's potential as a surveillance tool. Methods: We used bidirectional encoder representations from transformers (BERT) models to extract and normalize PCC symptom and condition terms from English posts on Twitter and Reddit. We compared 2 named entity recognition models and implemented a 2-step normalization task to map extracted terms to unique concepts in standardized terminology. The normalization steps were done using a semantic search approach with BERT biencoders. We evaluated the effectiveness of BERT models in extracting the terms using a human-annotated corpus and a proximity-based score. We also compared the validity and reliability of the extracted and normalized terms to a web-based survey with more than 3000 participants from several countries. Results: UmlsBERT-Clinical had the highest accuracy in predicting entities closest to those extracted by human annotators. Based on our findings, the top 3 most commonly occurring groups of PCC symptom and condition terms were systemic (such as fatigue), neuropsychiatric (such as anxiety and brain fog), and respiratory (such as shortness of breath). In addition, we also found novel symptom and condition terms that had not been categorized in previous studies, such as infection and pain. Regarding the co-occurring symptoms, the pair of fatigue and headaches was among the most co-occurring term pairs across both platforms. Based on the temporal analysis, the neuropsychiatric terms were the most prevalent, followed by the systemic category, on both social media platforms. Our spatial analysis concluded that 42% (10,938/26,247) of the analyzed terms included location information, with the majority coming from the United States, United Kingdom, and Canada. Conclusions: The outcome of our social media-derived pipeline is comparable with the results of peer-reviewed articles relevant to PCC symptoms. Overall, this study provides unique insights into patient-reported health outcomes of PCC and valuable information about the patient's journey that can help health care providers anticipate future needs. International registered report identifier (irrid): RR2-10.1101/2022.12.14.22283419.

Watch Where and How You Stick Pins When Playing With Voodoo Correlations

ESR studies on membrane fluidity of Chinese hamster ovary cells grown on microcarriers and in suspension.

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

Application of natural language processing to identify social needs from patient medical notes: development and assessment of a scalable, performant, and rule-based model in an integrated healthcare delivery system

[Cholestasis. Diagnostic value of immunoelectrophoresis].

Practical use case of natural language processing for observational clinical research data retrieval from electronic health records: AssistMED project

Identification of suicidal behavior among psychiatrically hospitalized adolescents using natural language processing and machine learning of electronic health records

Can natural language processing models extract and classify instances of interpersonal violence in mental healthcare electronic records: an applied evaluative study

Learning for Automotive Collision Avoidance and Autonomous Control

Generation and evaluation of artificial mental health records for Natural Language Processing

Improving ascertainment of suicidal ideation and suicide attempt with natural language processing

Enhancing Patient Experience by Automating and Transforming Free Text into Actionable Consumer Insights: A Natural Language Processing (NLP) Approach

Scalable information extraction from free text electronic health records using large language models

C-reactive protein: an early critical sign of clozapine-related myocarditis

Cohort Design and Natural Language Processing to Reduce Bias in Electronic Health Records Research: The Community Care Cohort Project

Automated recognition of functioning, activity and participation in COVID-19 from electronic patient records by natural language processing: a proof- of- concept

Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults

Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing

Applying natural language processing and machine learning techniques to patient experience feedback: a systematic review

Extracting social determinants of health from electronic health records using natural language processing: a systematic review

Using Social Media to Help Understand Patient-Reported Health Outcomes of Post-COVID-19 Condition: Natural Language Processing Approach