Abstract:Strategic reasoning is a complex yet essential capability for intelligent agents. It requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments. Unlike static reasoning tasks, success in these contexts depends on anticipating other agents' beliefs and actions while continuously adjusting strategies to achieve individual goals. LLMs and LLM agents often struggle with strategic reasoning due to the absence of a reasoning framework that enables them to dynamically infer others' perspectives and adapt to changing environments. Inspired by the Level-K framework from game theory and behavioral economics, which extends reasoning from simple reactions to structured strategic depth, we propose a novel framework: "K-Level Reasoning with Large Language Models (K-R)." This framework employs recursive mechanisms to enable LLMs to achieve varying levels of strategic depth, allowing agents to form higher order beliefs - beliefs about others' beliefs. We validate this framework through rigorous testing on four testbeds: two classical game theory problems and two social intelligence tasks. The results demonstrate the advantages of K-R in strategic reasoning. Our work presents the first recursive implementation of strategic depth in large language models (LLMs). It establishes a foundation for future research into theory of mind and strategic reasoning in LLMs.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of the lack of ability of large - language models (LLMs) in strategic reasoning. Specifically, the paper points out that existing LLMs and LLM agents have limited ability to dynamically adjust strategies in multi - agent environments, mainly because of the lack of a reasoning framework that enables them to dynamically infer the perspectives of other agents and adapt to changing environments. To overcome this challenge, the paper proposes a new framework - "K - level Reasoning with Large - Language Models (K - R)", which enables LLMs to reason at different levels of strategic depth through a recursive mechanism, thus forming higher - order beliefs (i.e., beliefs about other agents). ### Main contributions of the paper 1. **Introduction of the K - R framework**: This framework extends k - level thinking to LLMs, achieving flexible strategic reasoning through a recursive mechanism and being able to reason at different depths. 2. **Extensive experimental verification**: The paper conducts rigorous tests on four test platforms, including two classic game - theory problems and two social - intelligence tasks, demonstrating the significant advantages of the K - R framework in terms of flexibility and effectiveness. 3. **Theoretical analysis**: Through theoretical analysis, the paper confirms that the K - R framework can establish higher - order beliefs and enhance strategic - reasoning ability, laying the foundation for future research in the fields of theory of mind and strategic reasoning. ### Specific problems and solutions - **Problem**: Existing LLMs have limited ability to dynamically adjust strategies in multi - agent environments and are difficult to form higher - order beliefs. - **Solutions**: - **K - R framework**: Through a recursive mechanism, enables LLMs to infer the behaviors and beliefs of other agents layer by layer. - **Experimental verification**: Verifies the effectiveness of the K - R framework in multiple tasks, including: - **Guess 80% of the Average (G0.8A)**: Players need to choose a number that is closest to 80% of the average number chosen by all players. - **Survival Auction Game (SAG)**: Players need to bid for water resources to maintain health points in order to survive during a drought. - **Negotiation Task (NEG)**: Two agents need to negotiate the distribution of three items to maximize their own utilities. - **SOTOPIA Benchmark**: Simulates complex social interactions and evaluates the social intelligence of agents. ### Experimental results - **G0.8A**: The K - R method is significantly superior to other methods in terms of the winning rate, reaching a winning rate of 0.65. - **SAG**: The K - R method performs best in terms of the average number of survival rounds, reaching 9.01 rounds. - **NEG**: The K - R method has a winning rate of 59.08% in the negotiation task, which is significantly higher than other methods. - **SOTOPIA**: The performance of the K - R method in multiple dimensions is comparable to that of the GPT - 4 model, showing its potential in social - intelligence tasks. ### Conclusion The K - R framework significantly enhances the strategic - reasoning ability of LLMs through a recursive mechanism, enabling them to more effectively dynamically adjust strategies in multi - agent environments. This framework not only has a solid theoretical foundation but also performs well in practical tasks, providing a new direction for future research on theory of mind and strategic reasoning.

K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

The Emergence of Strategic Reasoning of Large Language Models

LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models

Strategic Reasoning with Language Models

Adaptive-Solver Framework for Dynamic Strategy Selection in Large Language Model Reasoning

Cooperative Strategic Planning Enhances Reasoning Capabilities in Large Language Models

Enhance Reasoning for Large Language Models in the Game Werewolf

A Principled Framework for Knowledge-enhanced Large Language Model

LLMs for Relational Reasoning: How Far are We?

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge

Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data

GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents

Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs

Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond

Case Study: Testing Model Capabilities in Some Reasoning Tasks

A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models

Towards Reasoning in Large Language Models: A Survey

GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations

Zero, Finite, and Infinite Belief History of Theory of Mind Reasoning in Large Language Models