Abstract:Difficult-to-treat depression (DTD) has been proposed as a broader and more clinically comprehensive perspective on a person's depressive disorder where despite treatment, they continue to experience significant burden. We sought to develop a Large Language Model (LLM)-based tool capable of interrogating routinely-collected, narrative (free-text) electronic health record (EHR) data to locate published prognostic factors that capture the clinical syndrome of DTD. In this work, we use LLM-generated synthetic data (GPT3.5) and a Non-Maximum Suppression (NMS) algorithm to train a BERT-based span extraction model. The resulting model is then able to extract and label spans related to a variety of relevant positive and negative factors in real clinical data (i.e. spans of text that increase or decrease the likelihood of a patient matching the DTD syndrome). We show it is possible to obtain good overall performance (0.70 F1 across polarity) on real clinical data on a set of as many as 20 different factors, and high performance (0.85 F1 with 0.95 precision) on a subset of important DTD factors such as history of abuse, family history of affective disorder, illness severity and suicidality by training the model exclusively on synthetic data. Our results show promise for future healthcare applications especially in applications where traditionally, highly confidential medical data and human-expert annotation would normally be required.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the problem of detecting the clinical features of Difficult - to - Treat Depression (DTD). Specifically, the researchers have developed a tool based on the Large Language Model (LLM) to extract DTD - related clinical features from Electronic Health Records (EHR). These features include the patient's history, disease severity, suicidal tendencies, etc., and these factors are often difficult to be fully captured by structured data fields in traditional treatments. ### Main objectives 1. **Train the model with synthetic data**: The researchers use synthetic data (such as GPT3.5) generated by LLM to train a BERT - based segment extraction model to identify and label DTD - related clinical features. 2. **Improve the model's performance on real data**: By training the model on synthetic data, the researchers hope to achieve good performance on real clinical data, especially for some important DTD features, such as abuse history, family history of affective disorders, disease severity and suicidal tendencies. 3. **Reduce the dependence on highly confidential medical data and manual annotation**: By using synthetic data, the researchers hope to reduce the need for highly confidential medical data and expensive manual annotation, thereby reducing the cost of data acquisition and annotation. ### Method overview - **Data generation**: Use GPT3.5 to generate synthetic clinical notes and label them through specific prompts. - **Model training**: Use the generated synthetic data to train a BERT - based segment extraction model that can identify and label text segments related to DTD. - **Performance evaluation**: Evaluate the performance of the model on real clinical data, especially in terms of identifying important DTD features. ### Innovation points - **Application of synthetic data**: It is the first attempt to train a model using only synthetic data to reduce the dependence on real data. - **Multi - factor extraction**: Not only focus on binary classification tasks, but also on more complex segment extraction tasks to provide more detailed clinical information. - **Privacy protection**: By using synthetic data, privacy problems caused by directly using real patient data are effectively avoided. ### Conclusion This study shows the potential of using synthetic data to train models in identifying the clinical features of Difficult - to - Treat Depression, especially in reducing the cost of data acquisition and annotation. Future work can further optimize the model performance, expand the application scope, and provide more powerful support for clinical decision - making.

Detecting the Clinical Features of Difficult-to-Treat Depression using Synthetic Data from Large Language Models

From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models

Depression Detection on Social Media with Large Language Models

Using LLMs to Aid Annotation and Collection of Clinically-Enriched Data in Bipolar Disorder and Schizophrenia

Identifying Psychiatric Manifestations in Outpatients with Depression and Anxiety: A Large Language Model-Based Approach

Bespoke Large Language Models for Digital Triage Assistance in Mental Health Care

Enabling Early Health Care Intervention by Detecting Depression in Users of Web-Based Forums using Language Models: Longitudinal Analysis and Evaluation

Large language models to identify social determinants of health in electronic health records

Model development for bespoke large language models for digital triage assistance in mental health care

Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Data

Illuminate: A novel approach for depression detection with explainable analysis and proactive therapy using prompt engineering

Mental Disorders Detection in the Era of Large Language Models

InterMind: A Doctor-Patient-Family Interactive Depression Assessment System Empowered by Large Language Models

CALLM: Enhancing Clinical Interview Analysis through Data Augmentation with Large Language Models

Evaluating Large Language Models for Anxiety and Depression Classification using Counseling and Psychotherapy Transcripts

Transformative potential of Large Language Models in data mining on Electronic Health Records.

Using Large Language Models to Detect Depression From User-Generated Diary Text Data as a Novel Approach in Digital Mental Health Screening: Instrument Validation Study

DepreSym: A Depression Symptom Annotated Corpus and the Role of LLMs as Assessors of Psychological Markers

Leveraging LLMs for Translating and Classifying Mental Health Data

When LLMs Meets Acoustic Landmarks: An Efficient Approach to Integrate Speech into Large Language Models for Depression Detection