Abstract:Human interactions are deeply rooted in the interplay of thoughts, beliefs, and desires made possible by Theory of Mind (ToM): our cognitive ability to understand the mental states of ourselves and others. Although ToM may come naturally to us, emulating it presents a challenge to even the most advanced Large Language Models (LLMs). Recent improvements to LLMs' reasoning capabilities from simple yet effective prompting techniques such as Chain-of-Thought have seen limited applicability to ToM. In this paper, we turn to the prominent cognitive science theory "Simulation Theory" to bridge this gap. We introduce SimToM, a novel two-stage prompting framework inspired by Simulation Theory's notion of perspective-taking. To implement this idea on current ToM benchmarks, SimToM first filters context based on what the character in question knows before answering a question about their mental state. Our approach, which requires no additional training and minimal prompt-tuning, shows substantial improvement over existing methods, and our analysis reveals the importance of perspective-taking to Theory-of-Mind capabilities. Our findings suggest perspective-taking as a promising direction for future research into improving LLMs' ToM capabilities.

What problem does this paper attempt to address?

The problem this paper attempts to address is: how to improve the ability of large language models (LLMs) to understand the mental states of others, i.e., enhancing their "Theory of Mind" (ToM) capabilities. Specifically, although simple prompting techniques such as "Chain-of-Thought" (CoT) have significantly improved the reasoning abilities of LLMs in recent years, their effectiveness in Theory of Mind tasks remains limited. Therefore, the authors introduce a new two-stage prompting framework—SIMTOM, inspired by the "Simulation Theory" in cognitive science. This framework filters background information through "perspective-taking" before answering questions about mental states. This method does not require additional training and can significantly enhance the Theory of Mind capabilities of LLMs with minimal prompt adjustments. ### Main Contributions: 1. **Proposing the SIMTOM Framework**: A two-stage prompting framework that first involves perspective-taking and then answers questions based on the filtered information. 2. **Experimental Validation**: Extensive experiments on multiple benchmarks (such as BigTOM and ToMI) demonstrate that SIMTOM significantly improves the Theory of Mind capabilities of LLMs. 3. **Analysis and Discussion**: Through ablation studies and comparative analysis, the importance of perspective-taking in enhancing Theory of Mind capabilities is further explored, and future research directions are proposed. ### Background and Motivation: - **Importance of Theory of Mind**: Theory of Mind is fundamental to human cognition and social interaction, yet even the most advanced LLMs perform poorly on this task. - **Limitations of Existing Methods**: Existing prompting techniques like Chain-of-Thought, while effective for some tasks, have limited success in Theory of Mind tasks. - **Inspiration from Simulation Theory**: Humans answer Theory of Mind questions by first engaging in perspective-taking, understanding others' beliefs and goals from their viewpoint, and then answering the questions. ### Method Overview: 1. **Perspective-Taking**: Filter out information unknown to the character, retaining only what is known to them. 2. **Question Answering**: Answer questions based on the filtered information. ### Experimental Results: - **Significant Improvement**: SIMTOM significantly enhances the Theory of Mind capabilities of LLMs across multiple benchmarks, especially in false belief tasks. - **Ablation Studies**: Single-step prompts perform much worse than two-step prompts, highlighting the importance of the perspective-taking step. - **Human-Annotated Perspectives**: Using human-annotated perspective information, LLMs' performance approaches perfection, further proving the critical role of perspective-taking. ### Conclusion: By introducing the perspective-taking step, SIMTOM significantly improves the Theory of Mind capabilities of LLMs, providing new directions for future research. Future work can further explore how to enhance the perspective-taking abilities of LLMs to better simulate human Theory of Mind reasoning.

Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities

TimeToM: Temporal Space is the Key to Unlocking the Door of Large Language Models' Theory-of-Mind

PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models

Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models

How FaR Are Large Language Models From Agents with Theory-of-Mind?

Boosting Theory-of-Mind Performance in Large Language Models via Prompting

Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Tests

SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models

Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker

Through the Theory of Mind's Eye: Reading Minds with Multimodal Video Large Language Models

ToM-LM: Delegating Theory of Mind Reasoning to External Symbolic Executors in Large Language Models

Zero, Finite, and Infinite Belief History of Theory of Mind Reasoning in Large Language Models

Theory of Mind for Multi-Agent Collaboration via Large Language Models

Language Models Represent Beliefs of Self and Others

Violation of Expectation via Metacognitive Prompting Reduces Theory of Mind Prediction Error in Large Language Models

Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thought

Probing the Robustness of Theory of Mind in Large Language Models

Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Models

Vision Language Models See What You Want but not What You See