SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent

Jiarui Ji,Yang Li,Hongtao Liu,Zhicheng Du,Zhewei Wei,Weiran Shen,Qi Qi,Yankai Lin
2024-10-18
Abstract:Public scarce resource allocation plays a crucial role in economics as it directly influences the efficiency and equity in society. Traditional studies including theoretical model-based, empirical study-based and simulation-based methods encounter limitations due to the idealized assumption of complete information and individual rationality, as well as constraints posed by limited available data. In this work, we propose an innovative framework, SRAP-Agent (Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent), which integrates Large Language Models (LLMs) into economic simulations, aiming to bridge the gap between theoretical models and real-world dynamics. Using public housing allocation scenarios as a case study, we conduct extensive policy simulation experiments to verify the feasibility and effectiveness of the SRAP-Agent and employ the Policy Optimization Algorithm with certain optimization objectives. The source code can be found in <a class="link-external link-https" href="https://github.com/jijiarui-cather/SRAPAgent_Framework" rel="external noopener nofollow">this https URL</a>
Computation and Language
What problem does this paper attempt to address?
The paper attempts to address the issues of efficiency and fairness in the allocation of public scarce resources. Traditional research methods (including theoretical models, empirical studies, and simulation methods) face numerous limitations when dealing with such issues, such as assumptions of complete information and individual rationality, limited data, etc. Therefore, the authors propose an innovative framework, SRAP-Agent (Simulation and Optimization Agent for Scarce Resource Allocation Policies based on Large Language Models), aiming to bridge the gap between theoretical models and real-world dynamics by integrating large language models (LLMs) into economic simulations. Specifically, the main contributions of the paper are as follows: 1. **Proposing the SRAP-Agent framework**: This framework utilizes large language models (such as GPT-3.5 and GPT-4) to simulate participants' decision-making behaviors, particularly conducting extensive policy simulation experiments in the context of public housing allocation, verifying the feasibility and effectiveness of SRAP-Agent. 2. **Optimizing policy algorithms**: A policy optimization algorithm (POA) based on genetic algorithms is proposed to find the optimal policies with specific optimization goals (such as social satisfaction and social fairness). 3. **Experimental validation**: Through the case of public housing allocation, three-step experiments were conducted: Turing test, analysis of the impact of policy parameters on resource allocation, and finding the optimal policy using the POA algorithm. ### Main Findings: 1. **Decision-making behavior simulation**: Agents based on GPT-3.5 perform closer to actual human behavior in simulating human decision-making behavior, while GPT-4 shows stronger capabilities in strategic decision-making. 2. **Impact of queuing strategies**: Queuing strategies that prioritize disadvantaged groups can reduce the satisfaction gap between disadvantaged and non-disadvantaged groups. 3. **Effectiveness of POA**: The POA algorithm can generate robust policies after multiple iterations, and after 50 iterations, the policy evaluation metric f(π) improved by about 20%. ### Experimental Results: - **Social satisfaction**: The optimized policy π∗_s outperforms the existing actual policy in terms of social satisfaction. - **Social fairness**: The optimized policy π∗_f improves social fairness by 13.2% compared to the existing policy. - **Overall performance**: Although πKM performs best in terms of social welfare indicators, its performance in social fairness indicators is poor, showing an excessive focus on advantaged groups. In summary, this paper provides a new solution to the problem of public scarce resource allocation by introducing large language models and optimization algorithms, effectively improving the efficiency and fairness of policies.