Warmth and competence in human-agent cooperation

Kevin R. McKee,Xuechunzi Bai,Susan T. Fiske

DOI: https://doi.org/10.1007/s10458-024-09649-6

2024-05-09

Abstract:Interaction and cooperation with humans are overarching aspirations of artificial intelligence (AI) research. Recent studies demonstrate that AI agents trained with deep reinforcement learning are capable of collaborating with humans. These studies primarily evaluate human compatibility through "objective" metrics such as task performance, obscuring potential variation in the levels of trust and subjective preference that different agents garner. To better understand the factors shaping subjective preferences in human-agent cooperation, we train deep reinforcement learning agents in Coins, a two-player social dilemma. We recruit $N = 501$ participants for a human-agent cooperation study and measure their impressions of the agents they encounter. Participants' perceptions of warmth and competence predict their stated preferences for different agents, above and beyond objective performance metrics. Drawing inspiration from social science and biology research, we subsequently implement a new ``partner choice'' framework to elicit revealed preferences: after playing an episode with an agent, participants are asked whether they would like to play the next episode with the same agent or to play alone. As with stated preferences, social perception better predicts participants' revealed preferences than does objective performance. Given these results, we recommend human-agent interaction researchers routinely incorporate the measurement of social perception and subjective preferences into their studies.

Human-Computer Interaction,Computers and Society,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to better understand humans' social perception (especially warmth and competence) of AI agents and its impact on subjective preferences in human - machine collaboration. Although existing research has shown that AI agents trained by deep reinforcement learning can cooperate with humans to complete tasks, these studies mainly rely on "objective" indicators (such as task performance) to evaluate human compatibility, ignoring potential differences in trust and subjective preferences among different agents. This paper trains deep reinforcement learning agents to participate in a two - person social dilemma game named "Coins" and recruits 501 participants to conduct human - machine cooperation experiments, aiming to explore the following questions:

1. **How does social perception affect humans' subjective preferences for AI agents**: The study found that participants' perception of agents' warmth and competence can predict their preferences for different agents better than objective performance indicators.
2. **Can social perception predict humans' preferences in actual choices**: In addition to asking participants about their preferences, the study also introduced a "partner selection" framework, allowing participants to decide whether to continue to cooperate with the same agent or play the next game alone after interacting with the agent. The results show that social perception can also better predict participants' actual choice preferences.

Through the research on these questions, the author suggests that the measurement of social perception and subjective preferences should be routinely incorporated into human - machine interaction research to more comprehensively evaluate the human compatibility of AI agents.