Abstract:Addressing the disparity between forecasts and actual results can enable individuals to expand their thought processes and stimulate self-reflection, thus promoting accurate planning. In this research, we present **PreAct**, an agent framework that integrates **pre**diction, **rea**soning, and **act**ion. By utilizing the information derived from predictions, the large language model (LLM) agent can provide a wider range and more strategically focused reasoning. This leads to more efficient actions that aid the agent in accomplishing intricate tasks. Our experimental results show that PreAct surpasses the ReAct method in completing complex tasks and that PreAct's performance can be further improved when paired with other memory or selection strategy techniques. We presented the model with varying quantities of historical predictions and discovered that these predictions consistently enhance LLM <a class="link-external link-http" href="http://planning.The" rel="external noopener nofollow">this http URL</a> variances in single-step reasoning between PreAct and ReAct indicate that PreAct indeed has benefits in terms of diversity and strategic orientation over ReAct.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: How to enhance the planning ability of agents by predicting future observations, thereby improving their performance in completing complex tasks. Specifically, the researchers proposed a new framework named PreAct, aiming to integrate prediction, reasoning, and action to improve the performance of large language model (LLM) agents when handling complex tasks.
### Main Problem Analysis
1. **Limitations of Existing Methods**
- Current methods such as ReAct mainly rely on Chain - of - Thought (COT), that is, combining historical thinking, actions, and observations to make decisions. However, these methods usually generate direct causal reasoning paths and may repeatedly generate the same actions, which limits their effectiveness in tasks requiring complex relationships.
- Existing work focuses more on designing better state selection strategies and ignores optimizing action sampling methods. This results in insufficient improvement in the diversity and quality of action sampling methods.
2. **Proposal of PreAct**
- PreAct enhances the directional and strategic reasoning abilities of LLM agents by introducing the prediction of future observations. Specifically, PreAct requires the agent to predict possible observations and their corresponding measures after each action, thus helping the agent to explore and think more widely and comprehensively and better handle complex tasks.
- In this way, PreAct not only improves the diversity of planning but also enhances the directional strategy, enabling the agent to complete complex tasks more efficiently.
### Formula Representation
In the Markov decision process, the behavior and state transition of the agent can be represented as:
- Action policy: \( a_k=\pi_{\text{agent}}(o_{k - 1},\text{history}) \)
- Observation update: \( o_k=\pi_{\text{env}}(o_{k - 1},a_k) \)
where \( o_k \) is the observation result at the \( k \) - th step, \( a_k \) is the action at the \( k \) - th step, \( \pi_{\text{agent}} \) is the action policy of the agent, \( \pi_{\text{env}} \) is the state transition function of the environment, and `history` is historical information.
### Experimental Verification
To verify the effectiveness of PreAct, the researchers designed multiple experiments, including:
- **Performance Comparison in Different Modes**: Compare the performance of PreAct and ReAct on four datasets: Householding, Operating System, Database, and Lateral Thinking Puzzles.
- **Influence of Historical Prediction**: Analyze the influence of different numbers of historical predictions on reasoning performance.
- **Analysis of Intrinsic Reasons**: Explore the advantages of PreAct in reasoning diversity and directional strategy.
Through these experiments, the researchers proved that PreAct is superior to ReAct in most cases, and historical prediction has a continuous positive impact on the reasoning ability of the model.
### Conclusion
PreAct significantly enhances the reasoning diversity and directional strategy of LLM agents by introducing the prediction of future observations, thereby improving their ability to complete complex tasks. The proposal of this framework provides new ideas and methods for future agent planning and decision - making.