Abstract:Virtual counselors powered by large language models (LLMs) aim to create interactive support systems that effectively assist clients struggling with mental health challenges. To replicate counselor-client conversations, researchers have built an online mental health platform that allows professional counselors to provide clients with text-based counseling services for about an hour per session. Notwithstanding its effectiveness, challenges exist as human annotation is time-consuming, cost-intensive, privacy-protected, and not scalable. To address this issue and investigate the applicability of LLMs in psychological counseling conversation simulation, we propose a framework that employs two LLMs via role-playing for simulating counselor-client interactions. Our framework involves two LLMs, one acting as a client equipped with a specific and real-life user profile and the other playing the role of an experienced counselor, generating professional responses using integrative therapy techniques. We implement both the counselor and the client by zero-shot prompting the GPT-4 model. In order to assess the effectiveness of LLMs in simulating counselor-client interactions and understand the disparities between LLM- and human-generated conversations, we evaluate the synthetic data from various perspectives. We begin by assessing the client's performance through automatic evaluations. Next, we analyze and compare the disparities between dialogues generated by the LLM and those generated by professional counselors. Furthermore, we conduct extensive experiments to thoroughly examine the performance of our LLM-based counselor trained with synthetic interactive dialogues by benchmarking against state-of-the-art models for mental health.

What problem does this paper attempt to address?

The problem this paper attempts to address is how to use large language models (LLMs) to generate high-quality, human-like interactive dialogues in psychological counseling simulations, reducing reliance on manually annotated data, improving the efficiency and scalability of data generation, and protecting user privacy. ### Specific issues include: 1. **Data generation efficiency and cost issues**: Existing psychological counseling platforms require a large amount of human resources to generate dialogue data, which is not only time-consuming and costly but also involves privacy protection issues, making large-scale promotion difficult. 2. **Dialogue quality and authenticity issues**: Existing LLM-based methods often overlook the gradual and exploratory nature of real interactions when generating multi-turn dialogues, resulting in dialogues that lack realism and depth. 3. **Model performance evaluation issues**: How to effectively evaluate the performance of LLMs in simulating psychological counseling dialogues and compare them with human-generated dialogues to ensure the reliability and effectiveness of the models. ### Proposed solutions in the paper: - **Framework design**: Propose an LLM-to-LLM interaction framework based on role-playing, where one LLM simulates the client and another LLM simulates an experienced counselor. - **Data generation**: Generate high-quality simulated dialogue data through zero-shot prompting of the GPT-4 model. - **Evaluation methods**: Evaluate the generated dialogue data from multiple perspectives, including automatic evaluation of the client's role-playing effect, comparing LLM-generated dialogues with human-generated dialogues, and assessing the performance of the dialogue system through benchmark testing. ### Goals: - **Generate high-quality synthetic dialogue data**: Ensure that the generated dialogue data is close to real human dialogues in terms of quality and diversity. - **Improve the efficiency and scalability of data generation**: Reduce reliance on manually annotated data, making the data generation process more efficient and scalable. - **Protect user privacy**: Avoid using users' sensitive information directly by generating simulated data, thus protecting user privacy. Through these methods, the paper aims to provide a new, efficient solution for research and applications in the field of psychological counseling.

Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions

An LLM-based Simulation Framework for Embodied Conversational Agents in Psychological Counseling

Towards a Client-Centered Assessment of LLM Therapists by Client Simulation

Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs

PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling

Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational Interviewing

Therapy as an NLP Task: Psychologists' Comparison of LLMs and Human Peers in CBT

Response Generation for Cognitive Behavioral Therapy with Large Language Models: Comparative Study with Socratic Questioning

LLM-Based Empathetic Response Through Psychologist-Agent Debate

Leveraging Large Language Model as Simulated Patients for Clinical Education

Optimizing Psychological Counseling with Instruction-Tuned Large Language Models

ChatCounselor: A Large Language Models for Mental Health Support

VCounselor: A Psychological Intervention Chat Agent Based on a Knowledge-Enhanced Large Language Model

A Computational Framework for Behavioral Assessment of LLM Therapists

Chatting Up Attachment: Using LLMs to Predict Adult Bonds

Harnessing Large Language Models' Empathetic Response Generation Capabilities for Online Mental Health Counselling Support

LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices

PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluation

Evaluating the Efficacy of Interactive Language Therapy Based on LLM for High-Functioning Autistic Adolescent Psychological Counseling

LLM Roleplay: Simulating Human-Chatbot Interaction

Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant