Abstract:Large Language Models (LLMs) have shown remarkable abilities across various language tasks, but solving complex reasoning problems remains a challenge. While existing methods like Chain-of-Thought (CoT) and Tree-of-Thought (ToT) enhance reasoning by decomposing problems or structuring prompts, they typically perform a single pass of reasoning and may fail to revisit flawed paths, compromising accuracy. To address this, we propose a novel reasoning framework called Forest-of-Thought (FoT), which integrates multiple reasoning trees to leverage collective decision-making for solving complex logical problems. FoT utilizes sparse activation strategies to select the most relevant reasoning paths, improving both efficiency and accuracy. Additionally, we introduce a dynamic self-correction strategy that enables real-time error correction and learning from past mistakes, as well as consensus-guided decision making strategies to optimize correctness and computational resources. Experimental results demonstrate that the FoT framework, combined with these strategies, significantly enhances the reasoning capabilities of LLMs, enabling them to solve complex tasks with greater precision and efficiency.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to address the challenges of large - language models (LLMs) in handling complex reasoning problems. Although existing methods such as Chain - of - Thought (CoT) and Tree - of - Thought (ToT) have enhanced reasoning capabilities by decomposing problems or structuring prompts, they usually perform reasoning only once and do not re - examine defective paths, thus affecting accuracy. Specifically, the paper proposes a new reasoning framework - Forest - of - Thought (FoT) to deal with the following problems: 1. **Limitations of single - reasoning paths**: Existing methods such as CoT and ToT do not re - evaluate other possible paths after completing one reasoning, which may lead to missing better solutions. 2. **Lack of dynamic self - correction mechanisms**: Existing methods are unable to correct errors in real - time and learn from past mistakes. 3. **Efficient use of computational resources**: A method that can optimize computational resources is required to ensure that the reasoning process is both efficient and accurate. To overcome these problems, the FoT framework takes advantage of the benefits of collective decision - making by integrating multiple reasoning trees, selects the most relevant reasoning paths, and introduces dynamic self - correction strategies and consensus - guided decision - making strategies, thereby significantly improving the reasoning ability and accuracy of LLMs. #### Main contributions - **Multi - tree integrated reasoning**: By integrating multiple reasoning trees, use collective decision - making to handle complex logical reasoning tasks. - **Sparse activation strategy**: Select the most relevant reasoning paths to improve efficiency and accuracy. - **Dynamic self - correction strategy**: Achieve the ability to correct errors in real - time and learn from past mistakes. - **Consensus - guided decision - making strategy**: Optimize correctness and the use of computational resources. These improvements enable LLMs to have higher precision and efficiency when solving complex tasks.

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

When Do Program-of-Thought Works for Reasoning?

On the Empirical Complexity of Reasoning and Planning in LLMs

Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation

Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Models

Technical Report: Enhancing LLM Reasoning with Reward-guided Tree Search

Concise and Organized Perception Facilitates Reasoning in Large Language Models

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Empowering Multi-step Reasoning across Languages via Tree-of-Thoughts

RATT: A Thought Structure for Coherent and Correct LLM Reasoning

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs

Large Language Model Guided Tree-of-Thought

Measuring Faithfulness in Chain-of-Thought Reasoning

Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning

Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic

Towards Faithful Chain-of-Thought: Large Language Models are Bridging Reasoners

On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models

Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models

Tree of Problems: Improving structured problem solving with compositionality

Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation