Abstract:Loneliness, or the lack of fulfilling relationships, significantly impacts a person's mental and physical well-being and is prevalent worldwide. Previous research suggests that large language models (LLMs) may help mitigate loneliness. However, we argue that the use of widespread LLMs like ChatGPT is more prevalent--and riskier, as they are not designed for this purpose. To explore this, we analysed user interactions with ChatGPT, particularly those outside of its marketed use as task-oriented assistant. In dialogues classified as lonely, users frequently (37%) sought advice or validation, and received good engagement. However, ChatGPT failed in sensitive scenarios, like responding appropriately to suicidal ideation or trauma. We also observed a 35% higher incidence of toxic content, with women being 22 times more likely to be targeted than men. Our findings underscore ethical and legal questions about this technology, and note risks like radicalisation or further isolation. We conclude with recommendations for research and industry to address loneliness.
Computation and Language,Artificial Intelligence,Computers and Society,Human-Computer Interaction
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper explores the potential and risks of large - scale language models (LLMs) such as ChatGPT in alleviating loneliness. Specifically, the author studies the interactions between users and ChatGPT, especially those conversations that go beyond its market positioning as a task - oriented assistant and involve emotional support or the seeking of companionship.
#### Main problems:
1. **The impact of loneliness on health**: Loneliness (the lack of meaningful interpersonal relationships) seriously affects an individual's mental and physical health and is widespread globally. Research shows that loneliness is associated with an increased risk of dementia, depression, and overall mortality.
2. **The role of LLMs in alleviating loneliness**: Although some studies suggest that LLMs can alleviate loneliness by maintaining persuasive conversations, the author believes that widely used LLMs (such as ChatGPT) are not designed for this purpose and may therefore pose risks.
3. **Ethical and legal issues**: Since these models are not specifically designed to provide mental health support, but are actually used by users as emotional support tools in practice, this has raised ethical and legal issues regarding informed consent and liability.
4. **The risk of toxic content**: The study found that in conversations involving loneliness, the proportion of toxic content is significantly higher than in general conversations, especially toxic content targeting women and minors is more common.
5. **Inadequate ability to handle complex emotional problems**: When users seek help in dealing with more complex emotional problems (such as suicidal ideation or trauma coping), ChatGPT's performance is not satisfactory and fails to provide appropriate support or advice.
### Research methods:
- **Data sources**: The author used a dataset named WildChat, which contains 79,951 interaction records between users and ChatGPT.
- **Label classification**: Classify the conversations through GPT - 4, identify which conversations involve loneliness, and further analyze the content and characteristics of these conversations.
- **Quantitative and qualitative analysis**: Combine quantitative data to analyze the types and frequencies of conversations, and qualitatively analyze the conversation content of lonely users to explore the effectiveness and limitations of LLMs in alleviating loneliness.
### Main findings:
1. **Some conversations do help alleviate loneliness**: Approximately 8% of the conversations were classified as "lonely", and some users obtained emotional support through communicating with ChatGPT, and the conversation length was also longer than that of ordinary conversations.
2. **Improper handling of complex emotional problems**: For problems requiring professional intervention (such as suicidal ideation or trauma), ChatGPT's answers are often not appropriate, and sometimes even provide inappropriate or ineffective advice.
3. **Increased toxic content**: 55% of the conversations involving loneliness contain toxic content (violence, harm, or sex - related), which is much higher than 20% in general conversations. Women and minors are the main targets, and women are 22 times more likely to be targeted than men.
4. **Differences in conversation quality**: Conversations related to loneliness tend to be more confrontational. Although ChatGPT tries to avoid the escalation of conflicts, these conversations are usually longer than other types of conversations, indicating that its coping strategies are insufficient in some cases.
### Conclusion:
This study emphasizes the challenges of safely deploying LLMs on an open, global scale, especially in alleviating loneliness. Although LLMs can provide some help to those seeking companionship, there are also risks of exacerbating social isolation, inadvertently causing harm, or amplifying toxic behaviors. Therefore, the author calls on technology companies and the research community to pay more attention to these issues and propose corresponding ethical and legal recommendations to ensure the safe use of LLMs.
---
If you have more questions or need further information, please feel free to let me know!