Abstract:Objective: This paper evaluates the application of a natural language processing (NLP) model for extracting clinical text referring to interpersonal violence using electronic health records (EHRs) from a large mental healthcare provider. Design: A multidisciplinary team iteratively developed guidelines for annotating clinical text referring to violence. Keywords were used to generate a dataset which was annotated (ie, classified as affirmed, negated or irrelevant) for: presence of violence, patient status (ie, as perpetrator, witness and/or victim of violence) and violence type (domestic, physical and/or sexual). An NLP approach using a pretrained transformer model, BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) was fine-tuned on the annotated dataset and evaluated using 10-fold cross-validation. Setting: We used the Clinical Records Interactive Search (CRIS) database, comprising over 500 000 de-identified EHRs of patients within the South London and Maudsley NHS Foundation Trust, a specialist mental healthcare provider serving an urban catchment area. Participants: Searches of CRIS were carried out based on 17 predefined keywords. Randomly selected text fragments were taken from the results for each keyword, amounting to 3771 text fragments from the records of 2832 patients. Outcome measures: We estimated precision, recall and F1 score for each NLP model. We examined sociodemographic and clinical variables in patients giving rise to the text data, and frequencies for each annotated violence characteristic. Results: Binary classification models were developed for six labels (violence presence, perpetrator, victim, domestic, physical and sexual). Among annotations affirmed for the presence of any violence, 78% (1724) referred to physical violence, 61% (1350) referred to patients as perpetrator and 33% (731) to domestic violence. NLP models' precision ranged from 89% (perpetrator) to 98% (sexual); recall ranged from 89% (victim, perpetrator) to 97% (sexual). Conclusions: State of the art NLP models can extract and classify clinical text on violence from EHRs at acceptable levels of scale, efficiency and accuracy.

Generation and evaluation of artificial mental health records for Natural Language Processing

Generating Synthetic Mixed-Type Longitudinal Electronic Health Records for Artificial Intelligent Applications

Generation of Synthetic Electronic Medical Record Text

Are Synthetic Clinical Notes Useful for Real Natural Language Processing Tasks: A Case Study on Clinical Entity Recognition

Natural Language Processing for Structuring Clinical Text Data on Depression Using UK-CRIS

Behind the scenes: A medical natural language processing project

A Method for Generating Synthetic Electronic Medical Record Text

Natural language processing for mental health interventions: a systematic review and research framework

Investigating online activity in UK adolescent mental health patients: a feasibility study using a natural language processing approach for electronic health records

Does Synthetic Data Generation of LLMs Help Clinical Text Mining?

Deep Learning-Based Natural Language Processing for Screening Psychiatric Patients

Robust Privacy Amidst Innovation with Large Language Models Through a Critical Assessment of the Risks

Can natural language processing models extract and classify instances of interpersonal violence in mental healthcare electronic records: an applied evaluative study

Artificial Intelligence in Psychiatry: A New Paradigm

Generating Medical Assessments Using a Neural Network Model: Algorithm Development and Validation

Generating Synthetic Free-text Medical Records with Low Re-identification Risk using Masked Language Modeling

Improving Clinical NLP Performance through Language Model-Generated Synthetic Clinical Data

Zero-shot and Few-shot Generation Strategies for Artificial Clinical Records

Natural Language Programming in Medicine: Administering Evidence Based Clinical Workflows with Autonomous Agents Powered by Generative Large Language Models

Aspiring to Unintended Consequences of Natural Language Processing: A Review of Recent Developments in Clinical and Consumer-Generated Text Processing

Development of a Human Evaluation Framework and Correlation with Automated Metrics for Natural Language Generation of Medical Diagnoses