Abstract:This study explores the potential of large language models (LLMs) to conduct market experiments, aiming to understand their capability to comprehend competitive market dynamics. We model the behavior of market agents in a controlled experimental setting, assessing their ability to converge toward competitive equilibria. The results reveal the challenges current LLMs face in replicating the dynamic decision-making processes characteristic of human trading behavior. Unlike humans, LLMs lacked the capacity to achieve market equilibrium. The research demonstrates that while LLMs provide a valuable tool for scalable and reproducible market simulations, their current limitations necessitate further advancements to fully capture the complexities of market behavior. Future work that enhances dynamic learning capabilities and incorporates elements of behavioral economics could improve the effectiveness of LLMs in the economic domain, providing new insights into market dynamics and aiding in the refinement of economic policies.
What problem does this paper attempt to address?
The paper attempts to address the problem of evaluating the ability of large language models (LLMs) to simulate competitive market behavior, specifically whether they can achieve market equilibrium in double auction experiments. Specifically, the researchers designed an experiment to test whether LLMs can adjust their bids and asks over multiple rounds of trading, similar to human traders, and ultimately converge to the theoretical market equilibrium price.
### Research Background and Motivation
- **Market Equilibrium**: Market equilibrium refers to a state in a market where supply equals demand, maximizing both consumer surplus and producer surplus. This state is crucial for understanding real-world market operations.
- **Double Auction Mechanism**: A double auction is a market mechanism where buyers and sellers submit bids and asks, respectively. A transaction occurs when a buyer's bid is not lower than a seller's ask.
- **Existing Research**: Previous studies have validated through controlled experiments that human traders can gradually approach market equilibrium under the double auction mechanism. However, it is unclear whether LLMs possess the same capability.
### Research Methodology
- **Experimental Design**: The researchers designed a double auction experiment with 22 agents (11 buyers and 11 sellers), each randomly assigned the role of buyer or seller. Each agent submits bids or asks in each round of trading based on their budget or cost.
- **Experimental Steps**:
1. **Initialization Phase**: The system sends detailed double auction rules and current round information to each agent to ensure they understand the rules.
2. **Price Posting and Matching**: A random agent is selected to send different prompts based on their role, and their response is recorded. If a buyer's bid is not lower than a seller's ask, a transaction occurs, and the transaction price is recorded and fed back to all agents.
3. **Final Reminder**: If an agent does not respond to the initial prompt, the system sends a final reminder. If there is no response after three reminders, the trading day ends.
### Experimental Results
- **Price Convergence**: Compared to experiments with human traders, LLMs did not show a clear trend of converging to the market equilibrium price over multiple rounds of trading. Transaction prices fluctuated around the target equilibrium price but showed no signs of stable convergence.
- **Volatility**: The volatility of transaction prices was high, especially in the later rounds of trading, with a significant increase in the volatility coefficient.
- **Agent Strategy Adaptability**: LLMs lacked adaptive learning capabilities and could not adjust their bidding and asking strategies based on historical trading results.
### Conclusion
- **Main Findings**: Although LLMs provide a scalable and repeatable tool for market simulation, they currently lack dynamic adaptability and real-time feedback capabilities, preventing them from converging to market equilibrium like human traders.
- **Limitations**: LLMs lack adaptive learning, have rigid algorithms, limited market feedback integration, and lack psychological factors, which restrict their performance in economic simulations.
- **Future Directions**: Future research could explore enhancing the dynamic learning capabilities of LLMs and incorporating elements of behavioral economics to improve their effectiveness in economic simulations.
### Significance
- **Research Significance**: This study provides an initial assessment of the application of LLMs in economic simulations, revealing their current limitations and directions for improvement.
- **Application Prospects**: Improved LLMs could be used for modeling complex economic scenarios, such as policy intervention effects and changes in consumer behavior, reducing the need for large-scale human experiments. In the business field, enhanced LLMs could serve as virtual agents to simulate dynamic pricing, auction platforms, and market forecasting, helping companies optimize decision-making.