Joon Sung Park,Carolyn Q. Zou,Aaron Shaw,Benjamin Mako Hill,Carrie Cai,Meredith Ringel Morris,Robb Willer,Percy Liang,Michael S. Bernstein
Abstract:The promise of human behavioral simulation--general-purpose computational agents that replicate human behavior across domains--could enable broad applications in policymaking and social science. We present a novel agent architecture that simulates the attitudes and behaviors of 1,052 real individuals--applying large language models to qualitative interviews about their lives, then measuring how well these agents replicate the attitudes and behaviors of the individuals that they represent. The generative agents replicate participants' responses on the General Social Survey 85% as accurately as participants replicate their own answers two weeks later, and perform comparably in predicting personality traits and outcomes in experimental replications. Our architecture reduces accuracy biases across racial and ideological groups compared to agents given demographic descriptions. This work provides a foundation for new tools that can help investigate individual and collective behavior.
What problem does this paper attempt to address?
This paper aims to solve the problem of how to simulate the attitudes and behaviors of real individuals through the Generative Agent Architecture. Specifically, the research team has developed a new agent architecture that can replicate the attitudes and behaviors of 1,052 real individuals based on in - depth interview data, using large language models (LLMs). This method can not only improve the accuracy of simulation, but also reduce biases among different ethnic and ideological groups, thus providing new tools for studying individual and collective behaviors.
### Main Contributions
1. **Improve Simulation Accuracy**: By combining in - depth interviews and personal data, the generated agents can more accurately simulate human behaviors in multiple social, political or informational contexts, especially in predicting personality traits and experimental results.
2. **Reduce Bias**: Compared with methods that only use demographic descriptions, interview - based generative agents show lower biases among different ethnic and ideological groups.
3. **Verify the Method**: The research team used a variety of classic social science measurement tools (such as the General Social Survey (GSS), the Big Five Personality Inventory, behavioral economics games, etc.) to evaluate the performance of generative agents and compared it with the performance of human participants.
### Method Overview
- **Data Collection**: 1,052 American participants were recruited through stratified sampling for two - hour - long audio interviews, and their questionnaire and experimental data were collected.
- **Agent Generation**: Using the participants' interview data and large language models, agents that can simulate their attitudes and behaviors were generated.
- **Performance Evaluation**: The prediction accuracy of the agents was evaluated by comparing the performance of the generated agents with that of actual participants in multiple tasks. The main tasks include GSS, the Big Five Personality Inventory, behavioral economics games, and replication experiments of five social science studies.
### Results
- **GSS Prediction**: The prediction accuracy rate of the generated agents on GSS is 85%, which is significantly better than that of agents based on demographic information and character descriptions.
- **Big Five Personality Prediction**: The prediction correlation coefficient of the generated agents on the Big Five personality traits is 0.80, which is also better than other methods.
- **Behavioral Economics Games**: The prediction correlation coefficient of the generated agents in behavioral economics games is 0.66, showing good predictive ability.
- **Experimental Replication**: In the replication experiments of five social science studies, the generated agents can successfully replicate the results of four experiments, and the effect size is highly correlated with that of human participants (correlation coefficient = 0.98).
### Bias Analysis
- **Reduce Bias**: Interview - based generative agents show lower biases in different ethnic, gender and political ideological groups, especially in GSS, Big Five personality traits and behavioral economics games.
### Research Significance
This research provides new tools and methods for social science research. Through generative agents, individual and collective behaviors can be more accurately simulated, thus helping researchers test various interventions and theories and expand our understanding of complex social structures.