Abstract:World models, which encapsulate the dynamics of how actions affect environments, are foundational to the functioning of intelligent agents. In this work, we explore the potential of Large Language Models (LLMs) to operate as world models. Although LLMs are not inherently designed to model real-world dynamics, we show that they can be induced to perform two critical world model functions: determining the applicability of an action based on a given world state, and predicting the resulting world state upon action execution. This is achieved by fine-tuning two separate LLMs-one for precondition prediction and another for effect prediction-while leveraging synthetic data generation techniques. Through human-participant studies, we validate that the precondition and effect knowledge generated by our models aligns with human understanding of world dynamics. We also analyze the extent to which the world model trained on our synthetic data results in an inferred state space that supports the creation of action chains, a necessary property for planning.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper explores whether large language models (LLMs) can function as world models. Specifically, the paper attempts to address the following key issues: 1. **Determining Action Applicability**: Given a world state, determine whether a certain action can be executed. 2. **Predicting State Transitions**: Predict the new world state after executing a valid action. ### Background and Motivation World models are a core component of intelligent agents, describing how the environment changes with actions. Traditional world models are usually constructed through methods like reinforcement learning, but these methods often require a large amount of data and computational resources. In contrast, large language models (such as GPT-4, ChatGPT, etc.) are not specifically designed to simulate real-world dynamics, but they exhibit strong capabilities in handling natural language tasks. Therefore, researchers hope to explore whether these models can be trained and adjusted appropriately to possess the functionality of world models. ### Main Contributions 1. **Inducing LLMs to Become World Models**: The paper proposes a method to fine-tune two LLMs (one for precondition prediction and another for effect prediction) so that they can function like world models. 2. **Generating Synthetic Data**: To train these two models, the paper provides a method to use LLMs to generate synthetic precondition and effect data. 3. **Validating Effectiveness**: Through human participant studies and automated evaluations, the quality of the generated precondition and effect data, as well as the overall method's effectiveness, were validated. ### Method Overview 1. **Precondition/Effect Reasoning Module**: A module was designed to infer the preconditions and effects of actions by fine-tuning LLMs. 2. **Semantic Matching Module**: Another module was designed to match the inferred preconditions and effects with the current world state and update the world state. 3. **Applying the World Model**: By invoking the above two modules, effective action applicability prediction and state transition prediction were achieved. ### Evaluation The paper evaluates the proposed method from multiple aspects: 1. **Effectiveness of Global-Local Prompting Techniques**: Evaluated the techniques for generating high-quality precondition/effect datasets. 2. **Effectiveness of the Precondition/Effect Reasoning Module**: Evaluated the accuracy of the reasoning module in predicting preconditions and effects. 3. **Overall Effectiveness of the World Model**: Evaluated the performance of the world model in action applicability prediction and state transition prediction. ### Conclusion The paper successfully demonstrates that with appropriate training and adjustments, large language models can possess the functionality of world models, thereby playing an important role in the design of intelligent agents. This achievement provides new directions for future research, especially in application scenarios that require understanding and predicting complex world dynamics.

Making Large Language Models into World Models with Precondition and Effect Knowledge

Language Models Meet World Models: Embodied Experiences Enhance Language Models

Large Language Models and Knowledge Graphs: Opportunities and Challenges

Grounding Large Language Models In Embodied Environment With Imperfect World Models

Large Language Models with Controllable Working Memory

Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning

Evaluating World Models with LLM for Decision Making

Language-Guided World Models: A Model-Based Approach to AI Control

Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

From task structures to world models: What do LLMs know?

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems

Large Knowledge Model: Perspectives and Challenges

Supervised Knowledge Makes Large Language Models Better In-context Learners

LLM-State: Open World State Representation for Long-horizon Task Planning with Large Language Model

Theory of Mind for Multi-Agent Collaboration via Large Language Models

Enhancing Agent Learning through World Dynamics Modeling

Mental Modeling of Reinforcement Learning Agents by Language Models

WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents

On the Modeling Capabilities of Large Language Models for Sequential Decision Making

Large Language Models as Commonsense Knowledge for Large-Scale Task Planning