Evaluating the Experience of LGBTQ+ People Using Large Language Model Based Chatbots for Mental Health Support

Zilin Ma,Yiyang Mei,Yinru Long,Zhaoyuan Su,Krzysztof Z. Gajos
DOI: https://doi.org/10.1145/3613904.3642482
2024-02-14
Abstract:LGBTQ+ individuals are increasingly turning to chatbots powered by large language models (LLMs) to meet their mental health needs. However, little research has explored whether these chatbots can adequately and safely provide tailored support for this demographic. We interviewed 18 LGBTQ+ and 13 non-LGBTQ+ participants about their experiences with LLM-based chatbots for mental health needs. LGBTQ+ participants relied on these chatbots for mental health support, likely due to an absence of support in real life. Notably, while LLMs offer prompt support, they frequently fall short in grasping the nuances of LGBTQ-specific challenges. Although fine-tuning LLMs to address LGBTQ+ needs can be a step in the right direction, it isn't the panacea. The deeper issue is entrenched in societal discrimination. Consequently, we call on future researchers and designers to look beyond mere technical refinements and advocate for holistic strategies that confront and counteract the societal biases burdening the LGBTQ+ community.
Human-Computer Interaction
What problem does this paper attempt to address?
### The Problem the Paper Attempts to Address This paper aims to explore the experiences and effectiveness of large language model (LLM)-driven chatbots in providing mental health support to the LGBTQ+ community. Specifically: 1. **Mental Health Needs of the LGBTQ+ Community**: Research has found that LGBTQ+ individuals, due to a lack of support in real life, are increasingly turning to LLM-driven chatbots for mental health support. These chatbots can provide immediate support and accessibility but often struggle to understand the unique challenges and nuances specific to the LGBTQ+ community. 2. **Technical Limitations and Social Bias**: Although fine-tuning LLMs can partially improve their performance, it is not a fundamental solution. The deeper issue lies in social discrimination. Therefore, the authors call for future researchers and designers to not only focus on technical improvements but also advocate for comprehensive strategies to combat social bias, in order to better support the LGBTQ+ community. 3. **Research Methods and Results**: Through interviews with 18 LGBTQ+ participants and 13 non-LGBTQ+ participants, the paper reveals that while LLM-driven chatbots excel in certain areas (such as emotional support and social skills development), they still have significant shortcomings in understanding and addressing the specific needs of the LGBTQ+ community. In summary, this paper aims to evaluate the actual experiences of the LGBTQ+ community using LLM-driven chatbots for mental health support and to explore how these technologies can be improved to better meet their needs.