Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant

Cheng Kang,Daniel Novak,Katerina Urbanova,Yuqing Cheng,Yong Hu

DOI: https://doi.org/10.1109/ICASSPW62465.2024.10626529

2024-09-03

Abstract:Large language models (LLMs) have demonstrated impressive generalization capabilities on specific tasks with human-written instruction data. However, the limited quantity, diversity, and professional expertise of such instruction data raise concerns about the performance of LLMs in psychotherapy tasks when provided with domain-specific instructions. To address this, we firstly propose Domain-Specific Assistant Instructions based on AlexanderStreet therapy, and secondly, we use an adaption fine-tuning method and retrieval augmented generation method to improve pre-trained LLMs. Through quantitative evaluation of linguistic quality using automatic and human evaluation, we observe that pre-trained LLMs on Psychotherapy Assistant Instructions outperform state-of-the-art LLMs response baselines. Our Assistant-Instruction approach offers a half-annotation method to align pre-trained LLMs with instructions and provide pre-trained LLMs with more psychotherapy knowledge.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

This paper aims to address the issue of large language models (LLMs) performing poorly on psychotherapy tasks. Although these models have demonstrated impressive generalization capabilities on specific tasks, their performance on psychotherapy tasks is suboptimal due to the limited quantity, diversity, and specialization of instruction data in this domain. To tackle this challenge, the authors propose two main approaches: 1. **Domain-Specific Assistant Instructions**: These instructions are based on Alexander Street therapy and aim to improve the performance of LLMs in psychotherapy tasks. 2. **Adaptive Fine-Tuning and Retrieval-Enhanced Generation Methods**: These methods are used to improve pre-trained LLMs, enabling them to better understand and handle tasks in the psychotherapy domain. Through quantitative evaluation of language quality (including both automatic and human assessments), the study found that LLMs pre-trained with psychotherapy assistant instructions outperformed existing state-of-the-art LLM response baselines in multiple benchmarks. Additionally, the study proposed a semi-supervised method that allows pre-trained LLMs to align with instructions and provide more psychotherapy knowledge.

Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant

Optimizing Psychological Counseling with Instruction-Tuned Large Language Models

Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions

Advancing Conversational Psychotherapy: Integrating Privacy, Dual-Memory, and Domain Expertise with Large Language Models

Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapy

Can AI Relate: Testing Large Language Model Response for Mental Health Support

CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy

Enhancing AI-Driven Psychological Consultation: Layered Prompts with Large Language Models

VCounselor: A Psychological Intervention Chat Agent Based on a Knowledge-Enhanced Large Language Model

Fine-tuning a LLM using Reinforcement Learning from Human Feedback for a Therapy Chatbot Application

ChatCounselor: A Large Language Models for Mental Health Support

Chain-of-Interaction: Enhancing Large Language Models for Psychiatric Behavior Understanding by Dyadic Contexts

Evaluating the Efficacy of Interactive Language Therapy Based on LLM for High-Functioning Autistic Adolescent Psychological Counseling

Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy

Enhancing Clinical Accuracy of Medical Chatbots with Large Language Models

Supporting the Demand on Mental Health Services with AI-Based Conversational Large Language Models (LLMs)

Exploring the Efficacy of Robotic Assistants with ChatGPT and Claude in Enhancing ADHD Therapy: Innovating Treatment Paradigms

A Computational Framework for Behavioral Assessment of LLM Therapists

Towards Optimizing and Evaluating a Retrieval Augmented QA Chatbot using LLMs with Human in the Loop

Response Generation for Cognitive Behavioral Therapy with Large Language Models: Comparative Study with Socratic Questioning

LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluation