ESR studies on membrane fluidity of Chinese hamster ovary cells grown on microcarriers and in suspension.

C. Lai,L. Hopwood,H. Swartz

DOI: https://doi.org/10.1016/0014-4827(80)90022-1

IF: 4.145

1980-12-01

Experimental Cell Research

Abstract:

What problem does this paper attempt to address?

Facilitating clinical research through automation: Combining optical character recognition with natural language processing

Julie Hom,Janet Nikowitz,Rebecca Ottesen,Joyce C Niland

DOI: https://doi.org/10.1177/17407745221093621

2022-05-24

Clinical Trials

Abstract:Clinical Trials, Ahead of Print. Background/AimsPerformance status is crucial for most clinical research, as an eligibility criterion, a comorbidity covariate, or a trial endpoint. Yet information on performance status often is embedded as free text within a patient's electronic medical record, rather than coded directly, thereby making this concept extremely difficult to extract for research. Furthermore, performance status information frequently resides in outside reports, which are scanned into the electronic medical record along with thousands of clinic notes. The image format of scanned documents also is a major obstacle to the search and retrieval of information, as natural language processing cannot be applied to unstructured text within an image. We, therefore, utilized optical character recognition software to convert images to a searchable format, allowing the application of natural language processing to identify pertinent performance status data elements within scanned electronic medical records.MethodsOur study cohort consisted of 189 subjects diagnosed with diffuse large B-cell lymphoma for whom performance status was a required data element for analysis of prognostic factors related to recurrence and survival. Manual abstraction of performance status was previously conducted by a clinical Subject Matter Expert, serving as the gold standard. Leveraging our data warehouse, we extracted relevant scanned electronic medical record documents and applied optical character recognition to these images using the ABBYY FineReader software. The Linguamatics i2e natural language processing software was then used to run queries for performance status against the corpus of electronic medical record documents. We evaluated our optical character recognition/natural language processing pipeline for accuracy and reduction in data extraction effort.ResultsWe found that there was high accuracy and reduced time for extraction of performance status data by applying our optical character recognition/natural language processing pipeline. The transformed scanned documents from a random sample of patients yielded excellent precision, recall, and F score, with <1% incorrect results. Time savings from a second cohort showed that median time to review documents for patients with performance status data present was reduced by a third. The major time savings was in the review of those documents that in fact did not contain performance status information: median of 18 minutes versus 108 minutes for manual review, an 83% reduction in data abstraction effort.ConclusionBy applying this optical character recognition/natural language processing pipeline, we achieved significant operational improvement and reduced time for information retrieval to support clinical research. Our study demonstrated that optical character recognition software provides an effective mechanism to transform scanned electronic medical record images to allow the application of natural language processing, yielding highly accurate data abstraction. We conclude that our optical character recognition/natural language processing pipeline can greatly facilitate research data abstraction by providing a highly focused data review, eliminating unnecessary manual review of the entire chart, and thus freeing time for abstracting other data elements requiring more human interpretation.

medicine, research & experimental
Watch Where and How You Stick Pins When Playing With Voodoo Correlations

C. McCormick

DOI: https://doi.org/10.1080/00221309.2012.737872

2013-01-01

The Journal of General Psychology

Abstract:ABSTRACT The author comments on the (mis?)portrayal of her research in an article by Brand and Bradley (2012).
[Usefulness of the Michigan Alcoholism Screening Test (MAST) in Poland].

Z. Falicki,J. Karczewski,L. Wandzel,W. Chrzanowski

Psychiatria Polska

Abstract:
A Scoping Review of Publicly Available Language Tasks in Clinical Natural Language Processing

Yanjun Gao,Dmitriy Dligach,Leslie Christensen,Samuel Tesch,Ryan Laffin,Dongfang Xu,Timothy Miller,Ozlem Uzuner,Matthew M Churpek,Majid Afshar

DOI: https://doi.org/10.48550/arXiv.2112.05780

2021-12-08

Abstract:Objective: to provide a scoping review of papers on clinical natural language processing (NLP) tasks that use publicly available electronic health record data from a cohort of patients. Materials and Methods: We searched six databases, including biomedical research and computer science literature database. A round of title/abstract screening and full-text screening were conducted by two reviewers. Our method followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. Results: A total of 35 papers with 47 clinical NLP tasks met inclusion criteria between 2007 and 2021. We categorized the tasks by the type of NLP problems, including name entity recognition, summarization, and other NLP tasks. Some tasks were introduced with a topic of clinical decision support applications, such as substance abuse, phenotyping, cohort selection for clinical trial. We summarized the tasks by publication and dataset information. Discussion: The breadth of clinical NLP tasks keeps growing as the field of NLP evolves with advancements in language systems. However, gaps exist in divergent interests between general domain NLP community and clinical informatics community, and in generalizability of the data sources. We also identified issues in data selection and preparation including the lack of time-sensitive data, and invalidity of problem size and evaluation. Conclusions: The existing clinical NLP tasks cover a wide range of topics and the field will continue to grow and attract more attention from both general domain NLP and clinical informatics community. We encourage future work to incorporate multi-disciplinary collaboration, reporting transparency, and standardization in data preparation.

Computation and Language,Artificial Intelligence
Clinical Decision Support and Natural Language Processing in Medicine: Systematic Literature Review

Hans Eguia,Carlos Luis Sánchez-Bocanegra,Franco Vinciarelli,Fernando Alvarez-Lopez,Francesc Saigí-Rubió

DOI: https://doi.org/10.2196/55315

IF: 7.076

2024-10-01

Journal of Medical Internet Research

Abstract:Background: Ensuring access to accurate and verified information is essential for effective patient treatment and diagnosis. Although health workers rely on the internet for clinical data, there is a need for a more streamlined approach. Objective: This systematic review aims to assess the current state of artificial intelligence (AI) and natural language processing (NLP) techniques in health care to identify their potential use in electronic health records and automated information searches. Methods: A search was conducted in the PubMed, Embase, ScienceDirect, Scopus, and Web of Science online databases for articles published between January 2000 and April 2023. The only inclusion criteria were (1) original research articles and studies on the application of AI-based medical clinical decision support using NLP techniques and (2) publications in English. A Critical Appraisal Skills Programme tool was used to assess the quality of the studies. Results: The search yielded 707 articles, from which 26 studies were included (24 original articles and 2 systematic reviews). Of the evaluated articles, 21 (81%) explained the use of NLP as a source of data collection, 18 (69%) used electronic health records as a data source, and a further 8 (31%) were based on clinical data. Only 5 (19%) of the articles showed the use of combined strategies for NLP to obtain clinical data. In total, 16 (62%) articles presented stand-alone data review algorithms. Other studies (n=9, 35%) showed that the clinical decision support system alternative was also a way of displaying the information obtained for immediate clinical use. Conclusions: The use of NLP engines can effectively improve clinical decision systems' accuracy, while biphasic tools combining AI algorithms and human criteria may optimize clinical diagnosis and treatment flows. Trial Registration: PROSPERO CRD42022373386; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=373386

health care sciences & services,medical informatics
Natural Language Processing to Identify Abnormal Breast, Lung, and Cervical Cancer Screening Test Results from Unstructured Reports to Support Timely Follow-up.

Courtney J Diamond,John Laurentiev,Jie Yang,Amy Wint,Kimberly A Harris,Tin H Dang,Amrita Mecker,Emily B Carpenter,Anna N Tosteson,Adam Wright,Jennifer S Haas,Steven J Atlas,Li Zhou

DOI: https://doi.org/10.3233/shti220112

2022-01-01

Abstract:Cancer screening and timely follow-up of abnormal results can reduce mortality. One barrier to follow-up is the failure to identify abnormal results. While EHRs have coded results for certain tests, cancer screening results are often stored in free-text reports, which limit capabilities for automated decision support. As part of the multilevel Follow-up of Cancer Screening (mFOCUS) trial, we developed and implemented a natural language processing (NLP) tool to assist with real-time detection of abnormal cancer screening test results (including mammograms, low-dose chest CT scans, and Pap smears) and identification of gynecological follow-up for higher risk abnormalities (i.e. colposcopy) from free-text reports. We demonstrate the integration and implementation of NLP, within the mFOCUS system, to improve the follow-up of abnormal cancer screening results in a large integrated healthcare system. The NLP pipelines have detected scenarios when guideline-recommended care was not delivered, in part because the provider mis-identified the text-based result reports.
Development and Validation of an Artificial Intelligence System to Optimize Clinician Review of Patient Records

Ethan Andrew Chi,Gordon Chi,Cheuk To Tsui,Yan Jiang,Karolin Jarr,Chiraag V Kulkarni,Michael Zhang,Jin Long,Andrew Y Ng,Pranav Rajpurkar,Sidhartha R Sinha

DOI: https://doi.org/10.1001/jamanetworkopen.2021.17391

2021-07-01

Abstract:Importance: Physicians are required to work with rapidly growing amounts of medical data. Approximately 62% of time per patient is devoted to reviewing electronic health records (EHRs), with clinical data review being the most time-consuming portion. Objective: To determine whether an artificial intelligence (AI) system developed to organize and display new patient referral records would improve a clinician's ability to extract patient information compared with the current standard of care. Design, setting, and participants: In this prognostic study, an AI system was created to organize patient records and improve data retrieval. To evaluate the system on time and accuracy, a nonblinded, prospective study was conducted at a single academic medical center. Recruitment emails were sent to all physicians in the gastroenterology division, and 12 clinicians agreed to participate. Each of the clinicians participating in the study received 2 referral records: 1 AI-optimized patient record and 1 standard (non-AI-optimized) patient record. For each record, clinicians were asked 22 questions requiring them to search the assigned record for clinically relevant information. Clinicians reviewed records from June 1 to August 30, 2020. Main outcomes and measures: The time required to answer each question, along with accuracy, was measured for both records, with and without AI optimization. Participants were asked to assess overall satisfaction with the AI system, their preferred review method (AI-optimized vs standard), and other topics to assess clinical utility. Results: Twelve gastroenterology physicians/fellows completed the study. Compared with standard (non-AI-optimized) patient record review, the AI system saved first-time physician users 18% of the time used to answer the clinical questions (10.5 [95% CI, 8.5-12.6] vs 12.8 [95% CI, 9.4-16.2] minutes; P = .02). There was no significant decrease in accuracy when physicians retrieved important patient information (83.7% [95% CI, 79.3%-88.2%] with the AI-optimized vs 86.0% [95% CI, 81.8%-90.2%] without the AI-optimized record; P = .81). Survey responses from physicians were generally positive across all questions. Eleven of 12 physicians (92%) preferred the AI-optimized record review to standard review. Despite a learning curve pointed out by respondents, 11 of 12 physicians believed that the technology would save them time to assess new patient records and were interested in using this technology in their clinic. Conclusions and relevance: In this prognostic study, an AI system helped physicians extract relevant patient information in a shorter time while maintaining high accuracy. This finding is particularly germane to the ever-increasing amounts of medical data and increased stressors on clinicians. Increased user familiarity with the AI system, along with further enhancements in the system itself, hold promise to further improve physician data extraction from large quantities of patient health records.
Practical use case of natural language processing for observational clinical research data retrieval from electronic health records: AssistMED project

Cezary Maciejewski,Krzysztof Ozierański,Mikołaj Basza,Adam Barwiołek,Michalina Ciurla,Aleksandra Bożym,Maciej J Krajsman,Piotr Lodziński,Grzegorz Opolski,Marcin Grabowski,Andrzej Cacko,Paweł Balsam,Maciej J. Krajsman

DOI: https://doi.org/10.20452/pamw.16704

2024-03-19

Polish Archives of Internal Medicine

Abstract:INTRODUCTION: Electronic health records (EHR) contain data valuable for clinical research but in textual format, requiring encoding to databases by a human- a lengthy and costly process. Natural language processing (NLP) is a computational technique that allows text analysis.OBJECTIVES: To demonstrate a practical use case of NLP for a large retrospective study cohort characterization and compare it to a human retrieval.PATIENTS AND METHODS: Anonymized discharge documentation of 10314 patients from the cardiology tertiary care department was analyzed for inclusion in the CRAFT registry (NCT02987062) of patients with atrial fibrillation (AF). Extensive clinical characteristics regarding concomitant diseases, medications, daily dosage and echocardiography were collected manually and through NLP.RESULTS: There were 3030 and 3029 patients identified by human and NLP-based approaches, respectively, reflecting 99.93% accuracy of NLP in detecting AF. Comprehensive baseline patient characteristics by NLP was faster than human analysis (3 hours and 15 minutes vs 71 hours and 12 minutes). The calculated CHA2DS2VASc and HAS-BLED scores based on both methods did not differ (human vs NLP; median, IQR, P value): 3 (2-5) vs 3 (2-5) P = 0.74 and 1 (1-2) vs 1 (1-2) P = 0.63. For most data, an almost perfect agreement between NLP and human retrieved characteristics was found; daily dosage identification was the least accurate NLP feature. Similar conclusions on cohort characteristics would be made; however, daily dosage detection for some drug groups would require additional human validation in the NLP-based cohort.CONCLUSIONS: NLP utilization on EHR may accelerate acquisition and provide accurate data for a retrospective study.

medicine, general & internal
A query interface for clinical research with Chinese electronic health record using Natural Language Processing

Mengyang Li,Hailing Cai,Yani Chen,Xudong Lu,Huilong Duan

DOI: https://doi.org/10.21203/rs.3.rs-1664677/v1

2022-01-01

Abstract:Abstract Background: The recruitment of clinical trials is a challenging task. Traditionally, the time-consuming task is accomplished manually or assisted by form-based tools. Electronic health record (EHR) contains comprehensive information which can speed up the process. The development of natural language processing (NLP) makes it possible to reduce manual effort. To our knowledge, there are almost no query interfaces based on NLP techniques for Chinese EHR.Methods: A query interface based on NLP was developed. Firstly, we collected and annotated eligibility criteria(EC) from the Chinese Clinical Trial Registry. Then, an information extraction(IE) system was developed to parse them including named entity recognition(NER), relation classification(RC), and concept matching(CM). Next, the extracted information was post-processed(PP) to generate formal queries. Finally, we built a natural language query interface based on the system.Results: 4691 stroke-related EC were collected for annotation. 91.23% F1 score was achieved for the NER task. For the RC task, 92.75% F1 score was achieved. For the CM task, 90.78% accuracy was achieved and an evaluation result showed 42.42% of entities from NER results can be matched with EHR. And a natural language query interface has been implemented and applied in clinical research about stroke. Conclusions: We build a query interface for Chinese EHR based on NLP techniques. The proposed information extraction pipeline can support medical professionals to reduce the information gap with minimal human effort when interacting with Chinese EHR.
Assessing the Performance of Clinical Natural Language Processing Systems: Development of an Evaluation Methodology (Preprint)

Lea Canales,Sebastian Menke,Stephanie Marchesseau,Ariel D’Agostino,Carlos del Rio-Bermudez,Miren Taberna,Jorge Tello

DOI: https://doi.org/10.2196/preprints.20492

2020-05-20

Abstract:BACKGROUND Clinical natural language processing (cNLP) systems are of crucial importance due to their increasing capability in extracting clinically important information from free text contained in electronic health records (EHRs). The conversion of a nonstructured representation of a patient’s clinical history into a structured format enables medical doctors to generate clinical knowledge at a level that was not possible before. Finally, the interpretation of the insights gained provided by cNLP systems has a great potential in driving decisions about clinical practice. However, carrying out robust evaluations of those cNLP systems is a complex task that is hindered by a lack of standard guidance on how to systematically approach them. OBJECTIVE Our objective was to offer natural language processing (NLP) experts a methodology for the evaluation of cNLP systems to assist them in carrying out this task. By following the proposed phases, the robustness and representativeness of the performance metrics of their own cNLP systems can be assured. METHODS The proposed evaluation methodology comprised five phases: (1) the definition of the target population, (2) the statistical document collection, (3) the design of the annotation guidelines and annotation project, (4) the external annotations, and (5) the cNLP system performance evaluation. We presented the application of all phases to evaluate the performance of a cNLP system called “EHRead Technology” (developed by Savana, an international medical company), applied in a study on patients with asthma. As part of the evaluation methodology, we introduced the Sample Size Calculator for Evaluations (SLiCE), a software tool that calculates the number of documents needed to achieve a statistically useful and resourceful gold standard. RESULTS The application of the proposed evaluation methodology on a real use-case study of patients with asthma revealed the benefit of the different phases for cNLP system evaluations. By using SLiCE to adjust the number of documents needed, a meaningful and resourceful gold standard was created. In the presented use-case, using as little as 519 EHRs, it was possible to evaluate the performance of the cNLP system and obtain performance metrics for the primary variable within the expected CIs. CONCLUSIONS We showed that our evaluation methodology can offer guidance to NLP experts on how to approach the evaluation of their cNLP systems. By following the five phases, NLP experts can assure the robustness of their evaluation and avoid unnecessary investment of human and financial resources. Besides the theoretical guidance, we offer SLiCE as an easy-to-use, open-source Python library. CLINICALTRIAL
Assessing the Performance of Clinical Natural Language Processing Systems: Development of an Evaluation Methodology

Lea Canales,Sebastian Menke,Stephanie Marchesseau,Ariel D’Agostino,Carlos del Rio-Bermudez,Miren Taberna,Jorge Tello

DOI: https://doi.org/10.2196/20492

IF: 3.2

2021-07-23

JMIR Medical Informatics

Abstract:Background Clinical natural language processing (cNLP) systems are of crucial importance due to their increasing capability in extracting clinically important information from free text contained in electronic health records (EHRs). The conversion of a nonstructured representation of a patient’s clinical history into a structured format enables medical doctors to generate clinical knowledge at a level that was not possible before. Finally, the interpretation of the insights gained provided by cNLP systems has a great potential in driving decisions about clinical practice. However, carrying out robust evaluations of those cNLP systems is a complex task that is hindered by a lack of standard guidance on how to systematically approach them. Objective Our objective was to offer natural language processing (NLP) experts a methodology for the evaluation of cNLP systems to assist them in carrying out this task. By following the proposed phases, the robustness and representativeness of the performance metrics of their own cNLP systems can be assured. Methods The proposed evaluation methodology comprised five phases: (1) the definition of the target population, (2) the statistical document collection, (3) the design of the annotation guidelines and annotation project, (4) the external annotations, and (5) the cNLP system performance evaluation. We presented the application of all phases to evaluate the performance of a cNLP system called “EHRead Technology” (developed by Savana, an international medical company), applied in a study on patients with asthma. As part of the evaluation methodology, we introduced the Sample Size Calculator for Evaluations (SLiCE), a software tool that calculates the number of documents needed to achieve a statistically useful and resourceful gold standard. Results The application of the proposed evaluation methodology on a real use-case study of patients with asthma revealed the benefit of the different phases for cNLP system evaluations. By using SLiCE to adjust the number of documents needed, a meaningful and resourceful gold standard was created. In the presented use-case, using as little as 519 EHRs, it was possible to evaluate the performance of the cNLP system and obtain performance metrics for the primary variable within the expected CIs. Conclusions We showed that our evaluation methodology can offer guidance to NLP experts on how to approach the evaluation of their cNLP systems. By following the five phases, NLP experts can assure the robustness of their evaluation and avoid unnecessary investment of human and financial resources. Besides the theoretical guidance, we offer SLiCE as an easy-to-use, open-source Python library.

medical informatics
Barriers and Facilitators of Obtaining Social Determinants of Health of Patients with Cancer Through the Electronic Health Record Using Natural Language Processing Technology: Qualitative Feasibility Study with Stakeholder Interviews

Jordan Alpert,Hyehyun (Julia) Kim,Cara McDonnell,Yi Guo,Thomas J. George,Jiang Bian,Yonghui Wu

DOI: https://doi.org/10.2196/43059

2022-01-01

JMIR Formative Research

Abstract:Background Social determinants of health (SDoH), such as geographic neighborhoods, access to health care, education, and social structure, are important factors affecting people’s health and health outcomes. The SDoH of patients are scarcely documented in a discrete format in electronic health records (EHRs) but are often available in free-text clinical narratives such as physician notes. Innovative methods like natural language processing (NLP) are being developed to identify and extract SDoH from EHRs, but it is imperative that the input of key stakeholders is included as NLP systems are designed. Objective This study aims to understand the feasibility, challenges, and benefits of developing an NLP system to uncover SDoH from clinical narratives by conducting interviews with key stakeholders: (1) oncologists, (2) data analysts, (3) citizen scientists, and (4) patient navigators. Methods Individuals who frequently work with SDoH data were invited to participate in semistructured interviews. All interviews were recorded and subsequently transcribed. After coding transcripts and developing a codebook, the constant comparative method was used to generate themes. Results A total of 16 participants were interviewed (5 data analysts, 4 patient navigators, 4 physicians, and 3 citizen scientists). Three main themes emerged, accompanied by subthemes. The first theme, importance and approaches to obtaining SDoH, describes how every participant (n=16, 100%) regarded SDoH as important. In particular, proximity to the hospital and income levels were frequently relied upon. Communication about SDoH typically occurs during the initial conversation with the oncologist, but more personal information is often acquired by patient navigators. The second theme, SDoH exists in numerous forms, exemplified how SDoH arises during informal communication and can be difficult to enter into the EHR. The final theme, incorporating SDoH into health services research, addresses how more informed SDoH can be collected. One strategy is to empower patients so they are aware about the importance of SDoH, as well as employing NLP techniques to make narrative data available in a discrete format, which can provide oncologists with actionable data summaries. Conclusions Extracting SDoH from EHRs was considered valuable and necessary, but obstacles such as narrative data format can make the process difficult. NLP can be a potential solution, but as the technology is developed, it is important to consider how key stakeholders document SDoH, apply the NLP systems, and use the extracted SDoH in health outcome studies.
A survey on clinical natural language processing in the United Kingdom from 2007 to 2022

Honghan Wu,Minhong Wang,Jinge Wu,Farah Francis,Yun-Hsuan Chang,Alex Shavick,Hang Dong,Michael T. C. Poon,Natalie Fitzpatrick,Adam P. Levine,Luke T. Slater,Alex Handy,Andreas Karwath,Georgios V. Gkoutos,Claude Chelala,Anoop Dinesh Shah,Robert Stewart,Nigel Collier,Beatrice Alex,William Whiteley,Cathie Sudlow,Angus Roberts,Richard J. B. Dobson

DOI: https://doi.org/10.1038/s41746-022-00730-6

IF: 15.2

2022-12-22

npj Digital Medicine

Abstract:Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects ( n = 94; £ = 41.97 m) funded by UK funders or the European Union's funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019–2022 was 80 times that of 2007–2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP's great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.

health care sciences & services,medical informatics
Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults

Majid Afshar,Sabrina Adelaine,Felice Resnik,Marlon P Mundt,John Long,Margaret Leaf,Theodore Ampian,Graham J Wills,Benjamin Schnapp,Michael Chao,Randy Brown,Cara Joyce,Brihat Sharma,Dmitriy Dligach,Elizabeth S Burnside,Jane Mahoney,Matthew M Churpek,Brian W Patterson,Frank Liao

DOI: https://doi.org/10.2196/44977

2023-04-20

Abstract:Background: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. Objective: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. Methods: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. Results: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. Conclusions: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence-driven CDS. Trial registration: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480.
Cohort Design and Natural Language Processing to Reduce Bias in Electronic Health Records Research: The Community Care Cohort Project

Shaan Khurshid,Christopher Reeder,Lia X. Harrington,Pulkit Singh,Gopal Sarma,Samuel F. Friedman,Paolo Di Achille,Nathaniel Diamant,Jonathan W. Cunningham,Ashby C. Turner,Emily S. Lau,Julian S. Haimovich,Mostafa A. Al-Alusi,Xin Wang,Marcus D.R. Klarqvist,Jeffrey M. Ashburner,Christian Diedrich,Mercedeh Ghadessi,Johanna Mielke,Hanna M. Eilken,Alice McElhinney,Andrea Derix,Steven J. Atlas,Patrick T. Ellinor,Anthony A. Philippakis,Christopher D. Anderson,Jennifer E. Ho,Puneet Batra,Steven A. Lubitz

DOI: https://doi.org/10.1101/2021.05.26.21257872

2021-05-30

Abstract:ABSTRACT Background Electronic health records (EHRs) promise to enable broad-ranging discovery with power exceeding that of conventional research cohort studies. However, research using EHR datasets may be subject to selection bias, which can be compounded by missing data, limiting the generalizability of derived insights. Methods Mass General Brigham (MGB) is a large New England-based healthcare network comprising seven tertiary care and community hospitals with associated outpatient practices. Within an MGB-based EHR warehouse of >3.5 million individuals with at least one ambulatory care visit, we approximated a community-based cohort study by selectively sampling individuals longitudinally attending primary care practices between 2001-2018 (n=520,868), which we named the Community Care Cohort Project (C3PO). We also utilized pre-trained deep natural language processing (NLP) models to recover vital signs (i.e., height, weight, and blood pressure) from unstructured notes in the EHR. We assessed the validity of C3PO by deploying established risk models including the Pooled Cohort Equations (PCE) and the Cohorts for Aging and Genomic Epidemiology Atrial Fibrillation (CHARGE-AF) score, and compared model performance in C3PO to that observed within typical EHR Convenience Samples which included all individuals from the same parent EHR with sufficient data to calculate each score but without a requirement for longitudinal primary care. All analyses were facilitated by the JEDI Extractive Data Infrastructure pipeline which we designed to efficiently aggregate EHR data within a unified framework conducive to regular updates. Results C3PO includes 520,868 individuals (mean age 48 years, 61% women, median follow-up 7.2 years, median primary care visits per individual 13). Estimated using reports, C3PO contains over 2.9 million electrocardiograms, 450,000 echocardiograms, 12,000 cardiac magnetic resonance images, and 75 million narrative notes. Using tabular data alone, 286,009 individuals (54.9%) had all vital signs available at baseline, which increased to 358,411 (68.8%) after NLP recovery (31% reduction in missingness). Among individuals with both NLP and tabular data available, NLP-extracted and tabular vital signs obtained on the same day were highly correlated (e.g., Pearson r range 0.95-0.99, p<0.01 for all). Both the PCE models (c-index range 0.724-0.770) and CHARGE-AF (c-index 0.782, 95% 0.777-0.787) demonstrated good discrimination. As compared to the Convenience Samples, AF and MI/stroke incidence rates in C3PO were lower and calibration error was smaller for both PCE (integrated calibration index range 0.012-0.030 vs. 0.028-0.046) and CHARGE-AF (0.028 vs. 0.036). Conclusions Intentional sampling of individuals receiving regular ambulatory care and use of NLP to recover missing data have the potential to reduce bias in EHR research and maximize generalizability of insights.
A Patient-Screening Tool for Clinical Research Based on Electronic Health Records Using OpenEHR: Development Study

Mengyang Li,Hailing Cai,Shan Nan,Jialin Li,Xudong Lu,Huilong Duan

DOI: https://doi.org/10.2196/33192

IF: 3.2

2021-01-01

JMIR Medical Informatics

Abstract:BACKGROUND The widespread adoption of electronic health records (EHRs) has facilitated the secondary use of EHR data for clinical research. However, screening eligible patients from EHRs is a challenging task. The concepts in eligibility criteria are not completely matched with EHRs, especially derived concepts. The lack of high-level expression of Structured Query Language (SQL) makes it difficult and time consuming to express them. The openEHR Expression Language (EL) as a domain-specific language based on clinical information models shows promise to represent complex eligibility criteria. OBJECTIVE The study aims to develop a patient-screening tool based on EHRs for clinical research using openEHR to solve concept mismatch and improve query performance. METHODS A patient-screening tool based on EHRs using openEHR was proposed. It uses the advantages of information models and EL in openEHR to provide high-level expressions and improve query performance. First, openEHR archetypes and templates were chosen to define concepts called simple concepts directly from EHRs. Second, openEHR EL was used to generate derived concepts by combining simple concepts and constraints. Third, a hierarchical index corresponding to archetypes in Elasticsearch (ES) was generated to improve query performance for subqueries and join queries related to the derived concepts. Finally, we realized a patient-screening tool for clinical research. RESULTS In total, 500 sentences randomly selected from 4691 eligibility criteria in 389 clinical trials on stroke from the Chinese Clinical Trial Registry (ChiCTR) were evaluated. An openEHR-based clinical data repository (CDR) in a grade A tertiary hospital in China was considered as an experimental environment. Based on these, 589 medical concepts were found in the 500 sentences. Of them, 513 (87.1%) concepts could be represented, while the others could not be, because of a lack of information models and coarse-grained requirements. In addition, our case study on 6 queries demonstrated that our tool shows better query performance among 4 cases (66.67%). CONCLUSIONS We developed a patient-screening tool using openEHR. It not only helps solve concept mismatch but also improves query performance to reduce the burden on researchers. In addition, we demonstrated a promising solution for secondary use of EHR data using openEHR, which can be referenced by other researchers.
Integrating biomarkers into clinical predictive and prognostic paradigms for prostate cancer: The role of p53 status.

M. Ritter,K. Gilchrist,D. Jarrard,B. Verhoven

DOI: https://doi.org/10.1200/jco.2004.22.14_suppl.9604

IF: 45.3

Journal of Clinical Oncology

Abstract:9604 Background: Prognostic groupings using PSA, grade and stage still leave significant unexplained outcome variability. Biological heterogeneity is likely a factor, emphasizing the importance of integrating biomarker data into predictive and prognostic paradigms. Novel therapy targeting options might also be identified along the way. The status of one such biomarker, p53, could alter curative-intent radiotherapy outcomes based on its position in radioresponse pathways. We therefore identified a cohort of low-to-moderate risk, previously-treated radiotherapy patients and determined PSA failure rates as a function of pretreatment p53 tumor status. A prognostically matched surgery group was studied in parallel to test whether p53 had true predictive specificity for radiation outcome or was only a global prognostic factor. METHODS A total of 81 and 53 patients were identified who previously underwent a radical prostatectomy or radiotherapy(68-70 Gy), respectively, and who shared similarly favorable characteristics (PSA ≤ 20 ng/ml, Gleason score ≤ 7; no hormonal therapy before failure; 6 yr median follow-up). Abnormal p53 status (≥ 10% labeling index) was determined immunohistochemically in pretreatment biopsy specimens. RESULTS About 34% of both surgery and radiation patient biopsy specimens had abnormal p53 labeling. The two groups had very similar 5-year actuarial PSA failure rates of about 31%. Multivariate analyses including stage, grade and initial PSA revealed that abnormal p53 strongly correlated with actuarial PSA failure in radiotherapy (hazard ratio of 4.6; p < 0.001), but not in surgery patients (hazard ratio of 2.14; p = 0.22). CONCLUSIONS Abnormal pretreatment p53 status in favorable patients strongly and independently predicted for PSA failure after radiotherapy but not radical prostatectomy. This radiation-specific predictive power for outcome, if further validated, could significantly influence patient management. Beyond simple stratification, pretreatment p53 testing could identify a role for altered treatment - perhaps aggressive dose escalation or a novel p53 targeting strategy. No significant financial relationships to disclose.
Natural Language Processing in Electronic Health Records in Relation to Healthcare Decision-making: A Systematic Review

Elias Hossain,Rajib Rana,Niall Higgins,Jeffrey Soar,Prabal Datta Barua,Anthony R. Pisani,Ph.D,Kathryn Turner}

2023-06-22

Abstract:Background: Natural Language Processing (NLP) is widely used to extract clinical insights from Electronic Health Records (EHRs). However, the lack of annotated data, automated tools, and other challenges hinder the full utilisation of NLP for EHRs. Various Machine Learning (ML), Deep Learning (DL) and NLP techniques are studied and compared to understand the limitations and opportunities in this space comprehensively. Methodology: After screening 261 articles from 11 databases, we included 127 papers for full-text review covering seven categories of articles: 1) medical note classification, 2) clinical entity recognition, 3) text summarisation, 4) deep learning (DL) and transfer learning architecture, 5) information extraction, 6) Medical language translation and 7) other NLP applications. This study follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Result and Discussion: EHR was the most commonly used data type among the selected articles, and the datasets were primarily unstructured. Various ML and DL methods were used, with prediction or classification being the most common application of ML or DL. The most common use cases were: the International Classification of Diseases, Ninth Revision (ICD-9) classification, clinical note analysis, and named entity recognition (NER) for clinical descriptions and research on psychiatric disorders. Conclusion: We find that the adopted ML models were not adequately assessed. In addition, the data imbalance problem is quite important, yet we must find techniques to address this underlining problem. Future studies should address key limitations in studies, primarily identifying Lupus Nephritis, Suicide Attempts, perinatal self-harmed and ICD-9 classification.

Computation and Language,Computers and Society
Use of an ambient artificial intelligence tool to improve quality of clinical documentation

Jasmine Balloch,Shankar Sridharan,Geralyn Oldham,Jo Wray,Paul Gough,Robert Robinson,Neil J Sebire,Saleh Khalil,Elham Asgari,Christopher Tan,Andrew Taylor,Dominic Pimenta

DOI: https://doi.org/10.1016/j.fhj.2024.100157

2024-06-26

Abstract:Background: Electronic health records (EHRs) have contributed to increased workloads for clinicians. Ambient artificial intelligence (AI) tools offer potential solutions, aiming to streamline clinical documentation and alleviate cognitive strain on healthcare providers. Objective: To assess the clinical utility of an ambient AI tool in enhancing consultation experience and the completion of clinical documentation. Methods: Outpatient consultations were simulated with actors and clinicians, comparing the AI tool against standard EHR practices. Documentation was assessed by the Sheffield Assessment Instrument for Letters (SAIL). Clinician experience was measured through questionnaires and the NASA Task Load Index. Results: AI-produced documentation achieved higher SAIL scores, with consultations 26.3% shorter on average, without impacting patient interaction time. Clinicians reported an enhanced experience and reduced task load. Conclusions: The AI tool significantly improved documentation quality and operational efficiency in simulated consultations. Clinicians recognised its potential to improve note-taking processes, indicating promise for integration into healthcare practices.
Improving ascertainment of suicidal ideation and suicide attempt with natural language processing

Cosmin A. Bejan,Michael Ripperger,Drew Wilimitis,Ryan Ahmed,JooEun Kang,Katelyn Robinson,Theodore J. Morley,Douglas M. Ruderfer,Colin G. Walsh

DOI: https://doi.org/10.1038/s41598-022-19358-3

IF: 4.6

2022-09-09

Scientific Reports

Abstract:Methods relying on diagnostic codes to identify suicidal ideation and suicide attempt in Electronic Health Records (EHRs) at scale are suboptimal because suicide-related outcomes are heavily under-coded. We propose to improve the ascertainment of suicidal outcomes using natural language processing (NLP). We developed information retrieval methodologies to search over 200 million notes from the Vanderbilt EHR. Suicide query terms were extracted using word2vec. A weakly supervised approach was designed to label cases of suicidal outcomes. The NLP validation of the top 200 retrieved patients showed high performance for suicidal ideation (area under the receiver operator curve [AUROC]: 98.6, 95% confidence interval [CI] 97.1–99.5) and suicide attempt (AUROC: 97.3, 95% CI 95.2–98.7). Case extraction produced the best performance when combining NLP and diagnostic codes and when accounting for negated suicide expressions in notes. Overall, we demonstrated that scalable and accurate NLP methods can be developed to identify suicidal behavior in EHRs to enhance prevention efforts, predictive models, and precision medicine.

multidisciplinary sciences

ESR studies on membrane fluidity of Chinese hamster ovary cells grown on microcarriers and in suspension.

Facilitating clinical research through automation: Combining optical character recognition with natural language processing

Watch Where and How You Stick Pins When Playing With Voodoo Correlations

[Usefulness of the Michigan Alcoholism Screening Test (MAST) in Poland].

A Scoping Review of Publicly Available Language Tasks in Clinical Natural Language Processing

Clinical Decision Support and Natural Language Processing in Medicine: Systematic Literature Review

Natural Language Processing to Identify Abnormal Breast, Lung, and Cervical Cancer Screening Test Results from Unstructured Reports to Support Timely Follow-up.

Development and Validation of an Artificial Intelligence System to Optimize Clinician Review of Patient Records

Practical use case of natural language processing for observational clinical research data retrieval from electronic health records: AssistMED project

A query interface for clinical research with Chinese electronic health record using Natural Language Processing

Assessing the Performance of Clinical Natural Language Processing Systems: Development of an Evaluation Methodology (Preprint)

Assessing the Performance of Clinical Natural Language Processing Systems: Development of an Evaluation Methodology

Barriers and Facilitators of Obtaining Social Determinants of Health of Patients with Cancer Through the Electronic Health Record Using Natural Language Processing Technology: Qualitative Feasibility Study with Stakeholder Interviews

A survey on clinical natural language processing in the United Kingdom from 2007 to 2022

Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults

Cohort Design and Natural Language Processing to Reduce Bias in Electronic Health Records Research: The Community Care Cohort Project

A Patient-Screening Tool for Clinical Research Based on Electronic Health Records Using OpenEHR: Development Study

Integrating biomarkers into clinical predictive and prognostic paradigms for prostate cancer: The role of p53 status.

Natural Language Processing in Electronic Health Records in Relation to Healthcare Decision-making: A Systematic Review

Use of an ambient artificial intelligence tool to improve quality of clinical documentation

Improving ascertainment of suicidal ideation and suicide attempt with natural language processing