Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapy

Xin Sun,Jan de Wit,Zhuying Li,Jiahuan Pei,Abdallah El Ali,Jos A.Bosch

2024-11-11

Abstract:Chatbots or conversational agents (CAs) are increasingly used to improve access to digital psychotherapy. Many current systems rely on rigid, rule-based designs, heavily dependent on expert-crafted dialogue scripts for guiding therapeutic conversations. Although recent advances in large language models (LLMs) offer the potential for more flexible interactions, their lack of controllability and transparency poses significant challenges in sensitive areas like psychotherapy. In this work, we explored how aligning LLMs with expert-crafted scripts can enhance psychotherapeutic chatbot performance. Our comparative study showed that LLMs aligned with expert-crafted scripts through prompting and fine-tuning significantly outperformed both pure LLMs and rule-based chatbots, achieving a more effective balance between dialogue flexibility and adherence to therapeutic principles. Building on findings, we proposed ``Script-Strategy Aligned Generation (SSAG)'', a flexible alignment approach that reduces reliance on fully scripted content while enhancing LLMs' therapeutic adherence and controllability. In a 10-day field study, SSAG demonstrated performance comparable to full script alignment and outperformed rule-based chatbots, empirically supporting SSAG as an efficient approach for aligning LLMs with domain expertise. Our work advances LLM applications in psychotherapy by providing a controllable, adaptable, and scalable solution for digital interventions, reducing reliance on expert effort. It also provides a collaborative framework for domain experts and developers to efficiently build expertise-aligned chatbots, broadening access to psychotherapy and behavioral interventions.

Human-Computer Interaction,Artificial Intelligence

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is how to improve the controllability and transparency of large - language models (LLMs) in psychotherapy while maintaining the flexibility of the conversation. Specifically, the author proposes the "Script - Strategy Aligned Generation (SSAG)" method, aiming to enhance the performance of LLMs in psychotherapy by aligning with expert - written conversation scripts and treatment strategies. The paper mainly explores the following two research questions:

1. **RQ1**: How can LLM - based chatbots be aligned with expert - written conversation scripts to provide psychotherapy in behavioral interventions? By comparing three types of chatbots - rule - based chatbots, LLM chatbots using Script - Aligned Generation (SAG), and pure LLM chatbots - their performance in terms of language quality, conversation relevance, engagement, perceived empathy, MI consistency, and motivation enhancement is evaluated.

2. **RQ2**: How can LLMs be efficiently aligned with expert - written conversation scripts for psychotherapy? Specifically, two alignment methods are explored: strict alignment (through fine - tuning or prompting) and flexible alignment (through the proposed SSAG method). Through a 10 - day field study, the performance of LLM chatbots using SAG (through prompting) and SSAG methods is compared, with rule - based chatbots as the control group.

Through these two research questions, the paper verifies the effectiveness of the SSAG method, that is, while reducing the dependence on fully pre - written conversation scripts, it can effectively align LLMs with domain expertise, thereby providing more flexible, controllable, and efficient conversation support in psychotherapy.