Abstract:In contemporary society, the issue of psychological health has become increasingly prominent, characterized by the diversification, complexity, and universality of mental disorders. Cognitive Behavioral Therapy (CBT), currently the most influential and clinically effective psychological treatment method with no side effects, has limited coverage and poor quality in most countries. In recent years, researches on the recognition and intervention of emotional disorders using large language models (LLMs) have been validated, providing new possibilities for psychological assistance therapy. However, are LLMs truly possible to conduct cognitive behavioral therapy? Many concerns have been raised by mental health experts regarding the use of LLMs for therapy. Seeking to answer this question, we collected real CBT corpus from online video websites, designed and conducted a targeted automatic evaluation framework involving the evaluation of emotion tendency of generated text, structured dialogue pattern and proactive inquiry ability. For emotion tendency, we calculate the emotion tendency score of the CBT dialogue text generated by each model. For structured dialogue pattern, we use a diverse range of automatic evaluation metrics to compare speaking style, the ability to maintain consistency of topic and the use of technology in CBT between different models . As for inquiring to guide the patient, we utilize PQA (Proactive Questioning Ability) metric. We also evaluated the CBT ability of the LLM after integrating a CBT knowledge base to explore the help of introducing additional knowledge to enhance the model's CBT counseling ability. Four LLM variants with excellent performance on natural language processing are evaluated, and the experimental result shows the great potential of LLMs in psychological counseling realm, especially after combining with other technological means.

Can Large Language Models Replace Therapists? Evaluating Performance at Simple Cognitive Behavioral Therapy Tasks

Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy?

CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy

Applications of large language models in psychiatry: a systematic review

Therapy as an NLP Task: Psychologists' Comparison of LLMs and Human Peers in CBT

Large language models in psychiatry: Opportunities and challenges

Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided

Rethinking Large Language Models in Mental Health Applications

Can AI Relate: Testing Large Language Model Response for Mental Health Support

A Computational Framework for Behavioral Assessment of LLM Therapists

HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy

Challenges of Large Language Models for Mental Health Counseling

Using large language models in psychology

Beyond rating scales: With targeted evaluation, large language models are poised for psychological assessment

CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering

Evaluating the Efficacy of Interactive Language Therapy Based on LLM for High-Functioning Autistic Adolescent Psychological Counseling

Large language models as linguistic simulators and cognitive models in human research

Perils and opportunities in using large language models in psychological research

The Cultural Psychology of Large Language Models: Is ChatGPT a Holistic or Analytic Thinker?