The illusion of artificial inclusion

William Agnew,A. Stevie Bergman,Jennifer Chien,Mark Díaz,Seliem El-Sayed,Jaylen Pittman,Shakir Mohamed,Kevin R. McKee

DOI: https://doi.org/10.1145/3613904.3642703

2024-02-05

Abstract:Human participants play a central role in the development of modern artificial intelligence (AI) technology, in psychological science, and in user research. Recent advances in generative AI have attracted growing interest to the possibility of replacing human participants in these domains with AI surrogates. We survey several such "substitution proposals" to better understand the arguments for and against substituting human participants with modern generative AI. Our scoping review indicates that the recent wave of these proposals is motivated by goals such as reducing the costs of research and development work and increasing the diversity of collected data. However, these proposals ignore and ultimately conflict with foundational values of work with human participants: representation, inclusion, and understanding. This paper critically examines the principles and goals underlying human participation to help chart out paths for future work that truly centers and empowers participants.

Computers and Society

What problem does this paper attempt to address?

This paper attempts to explore the validity and potential problems of the proposal to replace the role of human participants in technology development and scientific research with large - language models (LLMs) and other generative artificial intelligence systems. Specifically, by reviewing and analyzing the existing "replacement proposals", the paper aims to understand the motivations behind these proposals and evaluate their conflicts with the core values of human - participation in research - representativeness, inclusiveness, and understanding.

### Main Motivations
The paper points out that the replacement proposals are mainly based on the following four motivations:

1. **Increasing the speed and scale of research and development**: Almost all of the surveyed proposals mention that using LLMs can significantly improve data - processing speed, reduce the time required for projects, and thus accelerate the research and development process.
2. **Reducing costs and dependence on manual labor**: Multiple proposals emphasize that LLMs are more cost - effective than recruiting and managing human participants and can reduce the need for manual labor.
3. **Enhancing the demographic diversity of data sets**: Some proposals believe that LLMs can simulate populations with different genders, races, income levels, etc., thereby enhancing the diversity and representativeness of data sets.
4. **Protecting participants from harm**: Some proposals suggest that using LLMs can prevent human participants from being exposed to high - risk or unethical research environments.

### Core Issues
Although the above - mentioned motivations seem reasonable, the paper points out that these replacement proposals have fundamental conflicts and challenges:

- **Representativeness**: The data generated by LLMs may not truly reflect the complexity and diversity of human beings, especially when it comes to social behavior and psychological research.
- **Inclusiveness and Empowerment**: The replacement proposals ignore the initiative and agency of human participants, weakening their sense of participation and influence in the research and development process.
- **Understanding**: Through interaction with human participants, researchers can gain deeper understanding and insight, while LLMs cannot provide this deep - level interactive experience.

### Conclusion
The paper emphasizes that although LLMs perform well in certain specific tasks, they cannot completely replace the role of human participants in research and development. Therefore, future research and development work should focus more on how to truly centralize and empower participants and their communities while using generative AI.

Through this analysis, the paper calls on researchers and developers to take action at every stage to ensure that the interests of human participants and communities are fully represented and respected.