A Study on the Performance of Generative Pre-trained Transformer (GPT) in Simulating Depressed Individuals on the Standardized Depressive Symptom Scale

Sijin Cai,Nanfeng Zhang,Jiaying Zhu,Yanjie Liu,Yongjin Zhou

2023-07-17

Abstract:Background: Depression is a common mental disorder with societal and economic burden. Current diagnosis relies on self-reports and assessment scales, which have reliability issues. Objective approaches are needed for diagnosing depression. Objective: Evaluate the potential of GPT technology in diagnosing depression. Assess its ability to simulate individuals with depression and investigate the influence of depression scales. Methods: Three depression-related assessment tools (HAMD-17, SDS, GDS-15) were used. Two experiments simulated GPT responses to normal individuals and individuals with depression. Compare GPT's responses with expected results, assess its understanding of depressive symptoms, and performance differences under different conditions. Results: GPT's performance in depression assessment was evaluated. It aligned with scoring criteria for both individuals with depression and normal individuals. Some performance differences were observed based on depression severity. GPT performed better on scales with higher sensitivity. Conclusion: GPT accurately simulates individuals with depression and normal individuals during depression-related assessments. Deviations occur when simulating different degrees of depression, limiting understanding of mild and moderate cases. GPT performs better on scales with higher sensitivity, indicating potential for developing more effective depression scales. GPT has important potential in depression assessment, supporting clinicians and patients.

Neurons and Cognition,Machine Learning

What problem does this paper attempt to address?

The paper primarily explores the potential and performance of Generative Pre-trained Transformer (GPT) technology in simulating the performance of standardized depression symptom scales for patients with depression. ### Research Background Depression is a common mental disorder that imposes a significant burden on both society and individuals. Current diagnostic methods mainly rely on self-reports and standardized assessment scales, which may have issues with subjectivity and reliability. Therefore, researchers are exploring more objective and reliable methods for diagnosing depression. ### Research Objectives - **Evaluate the potential of GPT technology in depression diagnosis**: Determine whether GPT can accurately simulate the thought patterns of normal individuals and patients with depression based on standardized depression symptom scale scores. - **Evaluate the simulation effect for patients with different severity levels of depression**: Determine whether GPT can accurately simulate the thought patterns of patients with mild, moderate, and severe depression based on standardized depression symptom scale scores. - **Explore the impact of depression scales with different sensitivities on the simulation effect**: Analyze how depression scales with different sensitivities affect GPT's simulation effect for patients with varying degrees of depression. ### Methods The study employed three depression-related assessment tools (Hamilton Depression Rating Scale [HAMD-17], Self-Rating Depression Scale [SDS], Geriatric Depression Scale [GDS-15]) and conducted two independent experiments to simulate GPT's responses for normal individuals and patients with different severity levels of depression, comparing the results with expected standards. ### Results - GPT can accurately simulate the performance of normal individuals and patients with depression when completing depression symptom scales. - The simulation effect of GPT varies for patients with different severity levels of depression, with certain biases particularly in simulating patients with mild and moderate depression. - The simulation effect of GPT is better when using high-sensitivity scales. ### Discussion - GPT technology shows good potential in simulating healthy individuals and patients with depression. - GPT needs further improvement in simulating patients with different severity levels of depression, especially in accurately simulating patients with mild and moderate depression. - High-sensitivity scales help improve the accuracy and consistency of GPT in simulating patients with depression. ### Conclusion GPT technology has significant application potential in understanding and reflecting depression symptoms, which can assist clinicians and patients in depression assessment. However, the accuracy of simulating patients with different severity levels of depression still needs improvement, particularly in understanding and reflecting the inner states of patients with mild and moderate depression. Additionally, the study points out that using high-sensitivity scales can significantly enhance the simulation effect of GPT. Future research should continue to explore scales designed for specific populations to obtain more comprehensive results.

A Study on the Performance of Generative Pre-trained Transformer (GPT) in Simulating Depressed Individuals on the Standardized Depressive Symptom Scale

Explaining GPT-4's Schema of Depression Using Machine Behavior Analysis

Evaluating cognitive performance: Traditional methods vs. ChatGPT

Assessing the Role of the Generative Pretrained Transformer (GPT) in Alzheimer's Disease Management: Comparative Study of Neurologist- and Artificial Intelligence–Generated Responses

Using Large Language Models to Detect Depression From User-Generated Diary Text Data as a Novel Approach in Digital Mental Health Screening: Instrument Validation Study

Evaluating the Efficacy of AI-Based Interactive Assessments Using Large Language Models for Depression Screening

Performance of Progressive Generations of GPT on an Exam Designed for Certifying Physicians as Certified Clinical Densitometrists

Advancing Mental Health Pre-Screening: A New Custom GPT for Psychological Distress Assessment

Leveraging ChatGPT to optimize depression intervention through explainable deep learning

Are Different Versions of ChatGPT's Ability Comparable to the Clinical Diagnosis Presented in Case Reports? A Descriptive Study

FPT-Former: A Flexible Parallel Transformer of Recognizing Depression by Using Audiovisual Expert-Knowledge-Based Multimodal Measures

Performance of ChatGPT on the Chinese Postgraduate Examination for Clinical Medicine: Survey Study

[Performance of generative pre-trained transformer-4 on the certification test for mental health management: A factorial design]

AI-Enhanced Mental Health Diagnosis: Leveraging Transformers for Early Detection of Depression Tendency in Textual Data

A Deep Learning Driven Simulation Analysis of the Emotional Profiles of Depression Based on Facial Expression Dynamics

Using deeply time-series semantics to assess depressive symptoms based on clinical interview speech

Automatic depression severity assessment with deep learning using parameter-efficient tuning

Assessing Generative Pretrained Transformers (GPT) in Clinical Decision-Making: Comparative Analysis of GPT-3.5 and GPT-4

Natural Language Processing for Depression Prediction on Sina Weibo: Method Study and Analysis

Performance of generative pre-trained transformers (GPTs) in Certification Examination of the College of Family Physicians of Canada