Children's Overtrust and Shifting Perspectives of Generative AI

Jaemarie Solyst,Ellia Yang,Shixian Xie,Jessica Hammer,Amy Ogan,Motahhare Eslami

2024-06-30

Abstract:The capabilities of generative AI (genAI) have dramatically increased in recent times, and there are opportunities for children to leverage new features for personal and school-related endeavors. However, while the future of genAI is taking form, there remain potentially harmful limitations, such as generation of outputs with misinformation and bias. We ran a workshop study focused on ChatGPT to explore middle school girls' (N = 26) attitudes and reasoning about how genAI works. We focused on girls who are often disproportionately impacted by algorithmic bias. We found that: (1) middle school girls were initially overtrusting of genAI, (2) deliberate exposure to the limitations and mistakes of generative AI shifted this overtrust to disillusionment about genAI capabilities, though they were still optimistic for future possibilities of genAI, and (3) their ideas about school policy were nuanced. This work informs how children think about genAI like ChatGPT and its integration in learning settings.

Human-Computer Interaction

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to understand the views of middle school students (especially female students) on generative artificial intelligence (genAI) and the process of their changes. Specifically, the research focuses on the following points:

1. **How do middle school students view text - based generative AI**: The research explores the initial views of middle school students on generative AI and examines how these views are affected when they are exposed to misinformation or prejudice generated by generative AI.

2. **How do middle school students understand the capabilities of generative AI and learn its limitations**: Through a series of activities, such as guessing games, the research observes whether middle school students can more accurately assess the capabilities and limitations of generative AI after being exposed to its incorrect outputs.

3. **What do middle school students think school classroom policies should do in response to generative AI**: The research also explores the attitudes of middle school students towards the use of generative AI in schools, including their views on fair use policies and concerns about possible academic misconduct (such as cheating).

Through these research questions, the paper aims to reveal the degree of understanding and trust of middle school students in generative AI, and how to help them establish a more reasonable cognitive model through educational intervention measures, so as to make better use of generative AI technology.