Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Shunyu Yao,Dian Yu,Jeffrey Zhao,Izhak Shafran,Thomas L. Griffiths,Yuan Cao,Karthik Narasimhan
2023-12-04
Abstract:Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role. To surmount these challenges, we introduce a new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of Thought approach to prompting language models, and enables exploration over coherent units of text (thoughts) that serve as intermediate steps toward problem solving. ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices. Our experiments show that ToT significantly enhances language models' problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. Code repo with all prompts: <a class="link-external link-https" href="https://github.com/princeton-nlp/tree-of-thought-llm" rel="external noopener nofollow">this https URL</a>.
Computation and Language,Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?
The problem this paper attempts to address is the poor performance of existing language models in tasks that require exploration, strategic foresight, or critical initial decisions. Specifically, existing language models primarily rely on a left-to-right word-by-word generation mechanism, which limits their ability to handle tasks that require multi-step reasoning and global decision-making. To address these issues, the paper introduces a new framework called "Tree of Thoughts" (ToT), which extends the popular "Chain of Thought" method. This framework enables language models to explore coherent units of text (referred to as "thoughts") that serve as intermediate steps in problem-solving. In this way, ToT allows language models to make more thoughtful decisions by considering multiple reasoning paths, self-evaluating choices, and performing foresight or backtracking when necessary. The paper validates the effectiveness of ToT through three new tasks that require non-trivial planning or search (24-point game, creative writing, and mini crossword puzzles) and demonstrates that ToT significantly enhances the problem-solving capabilities of language models in these tasks. For example, in the 24-point game, GPT-4 with chain of thought prompts solved only 4% of the tasks, while the ToT method achieved a success rate of 74%.