Abstract:This paper develops theory and algorithms for interacting large language model agents (LLMAs) using methods from statistical signal processing and microeconomics. While both fields are mature, their application to decision-making by interacting LLMAs remains unexplored. Motivated by Bayesian sentiment analysis on online platforms, we construct interpretable models and stochastic control algorithms that enable LLMAs to interact and perform Bayesian inference. Because interacting LLMAs learn from prior decisions and external inputs, they exhibit bias and herding behavior. Thus, developing interpretable models and stochastic control algorithms is essential to understand and mitigate these behaviors. This paper has three main results. First, we show using Bayesian revealed preferences from microeconomics that an individual LLMA satisfies the sufficient conditions for rationally inattentive (bounded rationality) utility maximization and, given an observation, the LLMA chooses an action that maximizes a regularized utility. Second, we utilize Bayesian social learning to construct interpretable models for LLMAs that interact sequentially with each other and the environment while performing Bayesian inference. Our models capture the herding behavior exhibited by interacting LLMAs. Third, we propose a stochastic control framework to delay herding and improve state estimation accuracy under two settings: (a) centrally controlled LLMAs and (b) autonomous LLMAs with incentives. Throughout the paper, we demonstrate the efficacy of our methods on real datasets for hate speech classification and product quality assessment, using open-source models like Mistral and closed-source models like ChatGPT. The main takeaway of this paper, based on substantial empirical analysis and mathematical formalism, is that LLMAs act as rationally bounded Bayesian agents that exhibit social learning when interacting.

Modeling Boundedly Rational Agents with Latent Inference Budgets

Amortized Bayesian Decision Making for simulation-based models

Latent State Estimation Helps UI Agents to Reason

LLM-driven Imitation of Subrational Behavior : Illusion or Reality?

Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies

Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models

A Bandit Model for Human-Machine Decision Making with Private Information and Opacity

A Theory of Bounded Inductive Rationality

Interacting Large Language Model Agents. Interpretable Models and Social Learning

Generalized Bayesian Inference for Scientific Simulators via Amortized Cost Estimation

Inverse decision-making using neural amortized Bayesian actors

Online Decision Deferral under Budget Constraints

On efficient computation in active inference

Optimal simulation-based Bayesian decisions

Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice

Amortizing intractable inference in large language models

Active Learning for Accurate Estimation of Linear Models

Inference Scaling vs Reasoning: An Empirical Analysis of Compute-Optimal LLM Problem-Solving

Bayesian Active Learning for Discrete Latent Variable Models

Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference