Abstract:Theory-of-mind (ToM) is the modeling of mental states (such as belief, desire, knowledge, perception) through recursive ("I think you think I think ...") type reasoning in order to plan one's action or anticipate others' action. Such reasoning forms the core of strategic analysis in the game-theoretic setting. Traditional analysis of rational behavior in games of complete information is centered on the axiom of "common knowledge," according to which all players know something to be true, know that all players know it to be true, know that all players know all players know it to be true, etc. Such axiom requires recursive modeling of players to the full depth, and seems to contradict human empirical behavior revealed by behavioral game literature. Here, I propose that such deviation from normative analysis may be due to players' building predictive mental models of their co-players based on experience and context without necessarily assuming a priori full rationality and common knowledge, rather than due to any lapse in "instrumental rationality" whereby players (and co-players) translate the predictions from their mental models to optimal choice. I investigate this mental model account of theory-of-mind reasoning by constructing a series of two-player, sequential-move matrix games all terminating in a maximal of three steps. By carefully designing payoff matrices, the depth of recursive reasoning (i.e., first-order ToM versus second-order ToM) can be contrasted based on participants' choice behavior in those games. Empirical findings support the idea that depth of ToM recursion (related to perspective-taking) and instrumental rationality (rational application of belief-desire to action) constitute separate processes. Finally, I present a theoretical analysis of repeated games, such as the Iterated Prisoner Dilemma, and show how mutual cooperation can arise as individually rational outcome due to expected future interaction with the opponent.

If more than Analytical Modeling is Needed to Predict Real Agents' Strategic Interaction

Incorporating Pragmatic Reasoning Communication into Emergent Language

Strategic Reasoning with Language Models

Human-Agent Decision-making: Combining Theory and Practice

GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents

Reasoning and interaction for social artificial intelligence

Reflexive theory-of-mind reasoning in games: from empirical evidence to modeling

Explaining Decisions of Agents in Mixed-Motive Games

Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions

Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMs

Multi-dimensional reasoning in competitive resource allocation games: Evidence from intra-team communication

Knowledge, Belief and Counterfactual Reasoning in Games

Strategy Representation and Reasoning for Incomplete Information Concurrent Games in the Situation Calculus

Modeling, Replicating, and Predicting Human Behavior: A Survey

Mental Models of AI Agents in a Cooperative Game Setting

Reconciling Rationality and Stochasticity: Rich Behavioral Models in Two-Player Games

Factorised Active Inference for Strategic Multi-Agent Interactions

Modeling Conceptual Understanding in Image Reference Games

A Multi-Agent Game for Studying Human Decision-Making

Language-based game theory in the age of artificial intelligence

Game Theoretical Interactions of Moving Agents