Abstract:Mental illness remains one of the most critical public health issues. Despite its importance, many mental health professionals highlight a disconnect between their training and actual real-world patient practice. To help bridge this gap, we propose PATIENT-{\Psi}, a novel patient simulation framework for cognitive behavior therapy (CBT) training. To build PATIENT-{\Psi}, we construct diverse patient cognitive models based on CBT principles and use large language models (LLMs) programmed with these cognitive models to act as a simulated therapy patient. We propose an interactive training scheme, PATIENT-{\Psi}-TRAINER, for mental health trainees to practice a key skill in CBT -- formulating the cognitive model of the patient -- through role-playing a therapy session with PATIENT-{\Psi}. To evaluate PATIENT-{\Psi}, we conducted a comprehensive user study of 13 mental health trainees and 20 experts. The results demonstrate that practice using PATIENT-{\Psi}-TRAINER enhances the perceived skill acquisition and confidence of the trainees beyond existing forms of training such as textbooks, videos, and role-play with non-patients. Based on the experts' perceptions, PATIENT-{\Psi} is perceived to be closer to real patient interactions than GPT-4, and PATIENT-{\Psi}-TRAINER holds strong promise to improve trainee competencies. Our code and data are released at \url{<a class="link-external link-https" href="https://github.com/ruiyiw/patient-psi" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the gap between the training of mental health professionals and actual clinical practice. Despite mental illness being a significant public health issue, many mental health professionals report a notable difference between the knowledge they acquire during training and the situations they face when dealing with patients. To bridge this gap, the researchers propose a novel patient simulation framework named PATIENT -Ψ for training in Cognitive Behavioral Therapy (CBT). Specifically, the paper addresses the following key issues: 1. **The gap between training and actual clinical practice**: Many mental health professionals indicate that their training does not adequately prepare them for the unpredictability and complexity of real patient interactions. 2. **Inadequacies of existing training methods**: Current training methods, such as textbooks, videos, and non-patient role-playing exercises, fail to realistically simulate the actual therapeutic process. 3. **Improving training effectiveness**: By using large language models (LLMs) and cognitive models to simulate patients, the researchers aim to provide a more effective training method, enabling trainees to better master key CBT skills, such as constructing cognitive models of patients. ### Solution To achieve the above goals, the researchers propose the following solutions: 1. **Constructing PATIENT -Ψ**: The researchers built diverse patient cognitive models based on CBT principles and programmed LLMs using these cognitive models to simulate patients during therapy sessions. 2. **Designing PATIENT -Ψ-TRAINER**: This is an interactive training framework where trainees can practice constructing cognitive models of patients through role-playing exercises with PATIENT -Ψ. The system displays the original cognitive model as a reference, allowing trainees to compare their results and receive feedback. 3. **Evaluating the effectiveness of PATIENT -Ψ**: The researchers conducted comprehensive user studies, including 13 trainees and 20 experts, to assess the realism of PATIENT -Ψ and the training effectiveness of PATIENT -Ψ-TRAINER. ### Main Contributions 1. **Proposing PATIENT -Ψ**: A novel simulated therapy patient that combines cognitive models based on psychological principles with LLMs. 2. **Designing PATIENT -Ψ-TRAINER**: An interactive training framework for trainees to practice constructing CBT cognitive models. 3. **Creating and releasing the PATIENT -Ψ-CM dataset**: Contains high-quality foundational CBT cognitive models curated by clinical psychologists. 4. **User study results**: The study shows that PATIENT -Ψ outperforms existing training methods in terms of realism, and PATIENT -Ψ-TRAINER excels in improving trainees' CBT skills and confidence. Through these contributions, the researchers hope to significantly enhance the training effectiveness of mental health professionals, better preparing them to face the challenges of actual clinical practice.

PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals

MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorders

Leveraging Large Language Model as Simulated Patients for Clinical Education

Can AI Relate: Testing Large Language Model Response for Mental Health Support

WundtGPT: Shaping Large Language Models To Be An Empathetic, Proactive Psychologist

LLMs Can Simulate Standardized Patients via Agent Coevolution

Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator

CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy

Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy?

AIPatient: Simulating Patients with EHRs and LLM Powered Agentic Workflow

Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles

Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data

InterMind: A Doctor-Patient-Family Interactive Depression Assessment System Empowered by Large Language Models

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

AI-Enhanced Cognitive Behavioral Therapy: Deep Learning and Large Language Models for Extracting Cognitive Pathways from Social Media Texts

Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy

A Comprehensive Evaluation of Large Language Models on Mental Illnesses

Psy-LLM: Scaling up Global Mental Health Psychological Services with AI-based Large Language Models

AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator

Challenges of Large Language Models for Mental Health Counseling

Model development for bespoke large language models for digital triage assistance in mental health care