ChatCounselor: A Large Language Models for Mental Health Support

June M. Liu,Donghao Li,He Cao,Tianhe Ren,Zeyi Liao,Jiamin Wu

DOI: https://doi.org/10.48550/arXiv.2309.15461

2023-09-27

Abstract:This paper presents ChatCounselor, a large language model (LLM) solution designed to provide mental health support. Unlike generic chatbots, ChatCounselor is distinguished by its foundation in real conversations between consulting clients and professional psychologists, enabling it to possess specialized knowledge and counseling skills in the field of psychology. The training dataset, Psych8k, was constructed from 260 in-depth interviews, each spanning an hour. To assess the quality of counseling responses, the counseling Bench was devised. Leveraging GPT-4 and meticulously crafted prompts based on seven metrics of psychological counseling assessment, the model underwent evaluation using a set of real-world counseling questions. Impressively, ChatCounselor surpasses existing open-source models in the counseling Bench and approaches the performance level of ChatGPT, showcasing the remarkable enhancement in model capability attained through high-quality domain-specific data.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address issues in the field of mental health support, particularly when professional psychological counselors are unable to provide timely assistance due to high costs, complex appointment processes, and limited resources. The research team developed a large-scale language model (LLM) named ChatCounselor, specifically designed to offer psychological counseling services. Unlike general chatbots, ChatCounselor is trained on actual psychological counseling dialogue data and possesses professional psychological knowledge and counseling skills.

The core issue mentioned in the paper is that existing large-scale language models like GPT-4 and PaLM mainly focus on general domains and are not specifically trained for psychological counseling. Therefore, these models may not perform ideally in providing psychological counseling services. To address this issue, the research team created a dataset named Psych8k, which includes dialogue data extracted from 260 in-depth interviews. By using GPT-4 to process and refine this data, they constructed 8,187 question-answer pairs.

Additionally, the research team designed an evaluation framework called CounselingBench, which includes a comprehensive set of evaluation metrics for LLM psychological counseling services and an automated evaluation process supported by GPT-4. Experimental results show that ChatCounselor outperforms existing open-source models on CounselingBench and approaches the level of ChatGPT. This indicates that fine-tuning pre-trained models with high-quality domain-specific data can significantly improve their performance on specific tasks.