Abstract:Robust therapeutic relationships between counselors and clients are fundamental to counseling effectiveness. The assessment of therapeutic alliance is well-established in traditional face-to-face therapy but may not directly translate to text-based settings. With millions of individuals seeking support through online text-based counseling, understanding the relationship in such contexts is crucial. In this paper, we present an automatic approach using large language models (LLMs) to understand the development of therapeutic alliance in text-based counseling. We adapt a theoretically grounded framework specifically to the context of online text-based counseling and develop comprehensive guidelines for characterizing the alliance. We collect a comprehensive counseling dataset and conduct multiple expert evaluations on a subset based on this framework. Our LLM-based approach, combined with guidelines and simultaneous extraction of supportive evidence underlying its predictions, demonstrates effectiveness in identifying the therapeutic alliance. Through further LLM-based evaluations on additional conversations, our findings underscore the challenges counselors face in cultivating strong online relationships with clients. Furthermore, we demonstrate the potential of LLM-based feedback mechanisms to enhance counselors' ability to build relationships, supported by a small-scale proof-of-concept.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the understanding and assessment of therapeutic alliance in online text counseling. Specifically: 1. **Background Issues**: - **Assessment of Therapeutic Alliance in Traditional Face-to-Face Counseling**: In traditional face-to-face psychological counseling services, the assessment of therapeutic alliance is already well-established. - **Challenges in Online Text Counseling**: With more people seeking support through online text counseling, understanding and assessing therapeutic alliance in this context has become an important research topic. Currently, methods and resources for assessing therapeutic alliance in online text counseling are relatively scarce. 2. **Research Objectives**: - **Automatic Understanding of Therapeutic Alliance**: The paper proposes an automatic method based on large language models (LLMs) to understand the development of therapeutic alliance in online text counseling. - **Adaptive Framework**: Researchers adapt existing theories and scales of therapeutic alliance to the context of online text counseling and develop detailed guidelines to describe various dimensions of therapeutic alliance. - **Data Collection and Evaluation**: Researchers collected a large-scale online text counseling dataset and conducted expert evaluations on a portion of it to validate the effectiveness of the proposed framework and guidelines. 3. **Main Contributions**: - **Automatic Assessment Method**: By combining detailed guidelines and the Chain-of-Thought (CoT) process, LLMs can more effectively identify and assess therapeutic alliance. - **Empirical Study**: Experimental results show that LLMs perform excellently in understanding therapeutic alliance, especially when combined with detailed guidelines and CoT. - **Feedback Mechanism**: The study also demonstrates the potential of an LLM-based feedback mechanism, which can help counselors better understand their relationship with clients and improve their relationship-building skills. ### Summary The main purpose of the paper is to automatically understand and assess therapeutic alliance in online text counseling using large language models (LLMs) and detailed guidelines, thereby providing timely and effective feedback to counselors and improving counseling outcomes.

Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs

Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions

Understanding Client Reactions in Online Mental Health Counseling

A Computational Framework for Behavioral Assessment of LLM Therapists

Towards a Client-Centered Assessment of LLM Therapists by Client Simulation

Therapy as an NLP Task: Psychologists' Comparison of LLMs and Human Peers in CBT

Evaluating the Efficacy of Interactive Language Therapy Based on LLM for High-Functioning Autistic Adolescent Psychological Counseling

Towards Automated Real-time Evaluation in Text-based Counseling

Do Large Language Models Align with Core Mental Health Counseling Competencies?

The Influence of Task and Group Disparities over Users' Attitudes Toward Using Large Language Models for Psychotherapy

Predicting the Big Five Personality Traits in Chinese Counselling Dialogues Using Large Language Models

COMPASS: Computational Mapping of Patient-Therapist Alliance Strategies with Language Modeling

Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities

PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling

Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy?

Optimizing Psychological Counseling with Instruction-Tuned Large Language Models

AffirmativeAI: Towards LGBTQ+ Friendly Audit Frameworks for Large Language Models

Supporting the Demand on Mental Health Services with AI-Based Conversational Large Language Models (LLMs)

An LLM-based Simulation Framework for Embodied Conversational Agents in Psychological Counseling

PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluation

Dr. GPT in Campus Counseling: Understanding Higher Education Students' Opinions on LLM-assisted Mental Health Services