Toward Safe Evolution of Artificial Intelligence (AI) based Conversational Agents to Support Adolescent Mental and Sexual Health Knowledge Discovery

Jinkyung Park,Vivek Singh,Pamela Wisniewski

2024-04-04

Abstract:Following the recent release of various Artificial Intelligence (AI) based Conversation Agents (CAs), adolescents are increasingly using CAs for interactive knowledge discovery on sensitive topics, including mental and sexual health topics. Exploring such sensitive topics through online search has been an essential part of adolescent development, and CAs can support their knowledge discovery on such topics through human-like dialogues. Yet, unintended risks have been documented with adolescents' interactions with AI-based CAs, such as being exposed to inappropriate content, false information, and/or being given advice that is detrimental to their mental and physical well-being (e.g., to self-harm). In this position paper, we discuss the current landscape and opportunities for CAs to support adolescents' mental and sexual health knowledge discovery. We also discuss some of the challenges related to ensuring the safety of adolescents when interacting with CAs regarding sexual and mental health topics. We call for a discourse on how to set guardrails for the safe evolution of AI-based CAs for adolescents.

Human-Computer Interaction,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to explore and address the following issues:

1. **Safety of Adolescents Using AI-Based Conversational Agents (CAs) for Sensitive Topic Knowledge Exploration**: With the release of various AI-based conversational agents (such as Microsoft Bing, Google Bard, and OpenAI's GPT-4), adolescents are increasingly using these systems to obtain information about mental health and sexual health. However, existing safe search methods can no longer meet the needs of this interactive knowledge discovery.

2. **Challenges of Existing Technologies**: The paper discusses two technical approaches to building conversational agents—rule-based methods and large language model (LLM)-based methods—and points out the challenges faced by both. Rule-based CAs, due to predefined keywords and command limitations, result in responses that are not personalized enough and lack a humanized way of communication; while LLM-based CAs, although capable of generating more natural conversations, also pose risks of outputting inappropriate or inaccurate information.

3. **Future Research Directions**: To ensure the safety of adolescents when using AI conversational agents, the authors suggest further research on how to develop content generation methods that are suitable for the adolescent age group and accurate. Additionally, there is a need to better understand the specific needs and concerns of adolescents in the design of AI systems and to evaluate the safety of existing systems.

Through the above analysis, the paper calls on all sectors of society to pay attention and jointly explore how to establish an effective set of protective measures to promote the safe development of AI-based conversational agents in supporting adolescents' exploration of mental health and sexual health knowledge.