Abstract:Background: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. Objective: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. Methods: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. Results: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. Conclusions: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence-driven CDS. Trial registration: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480.

Implementation of a Machine Learning Risk Prediction Model for Postpartum Depression in the Electronic Health Records

Development and validation of a machine learning algorithm for predicting the risk of postpartum depression among pregnant women

Using Electronic Health Records and Machine Learning to Predict Postpartum Depression.

Preparing for the bedside-optimizing a postpartum depression risk prediction model for clinical implementation in a health system

Development and validation of a machine learning‐based postpartum depression prediction model: A nationwide cohort study

Trustworthy Data and AI Environments for Clinical Prediction: Application to Crisis-Risk in People with Depression

Predicting Women with Postpartum Depression Symptoms Using Machine Learning Techniques

Machine Learning-Based Predictive Modeling of Postpartum Depression

Closing the Gap in High-Risk Pregnancy Care Using Machine Learning and Human-AI Collaboration

PARCCS: A Machine Learning Risk-Prediction Model for Acute Peripartum Cardiovascular Complications During Delivery Admissions

Design and development of a machine-learning-driven opioid overdose risk prediction tool integrated in electronic health records in primary care settings

Stratifying Risk for Postpartum Depression at Time of Hospital Discharge

Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records

Recent advancements in postpartum depression prediction through machine learning approaches: a systematic review

Unveiling the prevalence and risk factors of early stage postpartum depression: a hybrid deep learning approach

Machine Learning in Electronic Health Records: Identifying High-Risk Obstetric Patients Pre and During Labor

Harnessing Consumer Wearable Digital Biomarkers for Individualized Recognition of Postpartum Depression Using the All of Us Research Program Data Set: Cross-Sectional Study

Adolescent, parent, and provider attitudes toward a machine learning based clinical decision support system for selecting treatment for youth depression

Machine Learning Methods for Predicting Postpartum Depression: Scoping Review

Improving postpartum hemorrhage risk prediction using longitudinal electronic medical records

Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults