Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support

Ashish Sharma,Inna W. Lin,Adam S. Miner,David C. Atkins,Tim Althoff

DOI: https://doi.org/10.48550/arXiv.2203.15144

2022-03-29

Abstract:Advances in artificial intelligence (AI) are enabling systems that augment and collaborate with humans to perform simple, mechanistic tasks like scheduling meetings and grammar-checking text. However, such Human-AI collaboration poses challenges for more complex, creative tasks, such as carrying out empathic conversations, due to difficulties of AI systems in understanding complex human emotions and the open-ended nature of these tasks. Here, we focus on peer-to-peer mental health support, a setting in which empathy is critical for success, and examine how AI can collaborate with humans to facilitate peer empathy during textual, online supportive conversations. We develop Hailey, an AI-in-the-loop agent that provides just-in-time feedback to help participants who provide support (peer supporters) respond more empathically to those seeking help (support seekers). We evaluate Hailey in a non-clinical randomized controlled trial with real-world peer supporters on TalkLife (N=300), a large online peer-to-peer support platform. We show that our Human-AI collaboration approach leads to a 19.60% increase in conversational empathy between peers overall. Furthermore, we find a larger 38.88% increase in empathy within the subsample of peer supporters who self-identify as experiencing difficulty providing support. We systematically analyze the Human-AI collaboration patterns and find that peer supporters are able to use the AI feedback both directly and indirectly without becoming overly reliant on AI while reporting improved self-efficacy post-feedback. Our findings demonstrate the potential of feedback-driven, AI-in-the-loop writing systems to empower humans in open-ended, social, creative tasks such as empathic conversations.

Computation and Language,Human-Computer Interaction,Social and Information Networks

What problem does this paper attempt to address?

The paper aims to address the issue of how to facilitate more empathetic conversations in text-based peer-to-peer mental health support through collaboration between artificial intelligence (AI) and humans. Specifically, the research focuses on the following points:

1. **Enhancing Empathy Expression**: On mental health support platforms, many untrained supporters find it challenging to write empathetic responses. The research developed an AI-assisted system named HAILEY, designed to provide these supporters with instant feedback to help them write more empathetic responses.

2. **Evaluating AI Assistance Effectiveness**: Through a randomized controlled trial, the research evaluated the effectiveness of the HAILEY system in real-world applications. The results showed that the responses written by supporters in the AI-assisted group exhibited higher levels of empathy compared to those written by the human-only group.

3. **Effect Analysis for Specific Groups**: For supporters who reported difficulties in writing supportive responses, the assistance from the HAILEY system was particularly significant, with their empathy expression improving by 38.88%, which is much higher than the improvement seen in supporters who did not report such difficulties.

4. **User Interaction Pattern Analysis**: The research also analyzed different patterns of interaction between participants and the AI, finding that most participants were able to effectively utilize AI suggestions without becoming overly reliant on the AI, instead using the suggestions as a source of inspiration for creative application.

5. **System Acceptance**: Finally, the research investigated participants' perceptions of the HAILEY system. Most participants felt that this instant feedback system helped improve their support capabilities and expressed a desire for such systems to be implemented on actual mental health support platforms.

In summary, the research aims to demonstrate how artificial intelligence can collaborate with humans in open, social, and creative tasks, particularly in the field of mental health support that requires a high degree of empathy, thereby enhancing the overall quality of support.