Abstract:Chain-of-Thought (CoT) prompting enables large language models to solve complex reasoning problems by generating intermediate steps. However, confined by its inherent single-pass and sequential generation process, CoT heavily relies on the initial decisions, causing errors in early steps to accumulate and impact the final answers. In contrast, humans adopt recursive thinking when tackling complex reasoning problems, i.e., iteratively breaking the original problem into approachable sub-problems and aggregating their answers to resolve the original one. Inspired by the human cognitive process, we propose SOCRATIC QUESTIONING, a divide-and-conquer style algorithm that mimics the recursive thinking process. Specifically, SOCRATIC QUESTIONING leverages large language models to raise and answer sub-questions until collecting enough information to tackle the original question. Unlike CoT, SOCRATIC QUESTIONING explicitly navigates the thinking space, stimulates effective recursive thinking, and is more robust towards errors in the thinking process. Extensive experiments on several complex reasoning tasks, including MMLU, MATH, LogiQA, and visual question-answering demonstrate significant performance improvements over the state-of-the-art prompting methods, such as CoT, and Tree-of-Thought. The qualitative analysis clearly shows that the intermediate reasoning steps elicited by SOCRATIC QUESTIONING are similar to humans' recursively thinking process of complex reasoning problems.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to address the limitations of large - language models (LLMs) when dealing with complex reasoning problems. Specifically, the traditional Chain - of - Thought (CoT) method has the following issues: 1. **Single - pass and sequential generation process**: CoT relies on a single - pass and sequential generation process, which means that each step is generated based on the previous step. If errors occur in the early steps, these errors will accumulate and affect the accuracy of the final answer. 2. **Lack of the ability to optimize the generated reasoning paths**: CoT is unable to correct or optimize the already - generated reasoning paths, resulting in difficulty in correction once an error occurs. 3. **Limited ability to decompose complex problems**: When dealing with very complex reasoning problems, CoT may not be able to effectively decompose the problems, thus leading to the accumulation of errors in the reasoning process. To solve these problems, the paper proposes a new algorithm - **SOCRATIC QUESTIONING**. This algorithm draws on the recursive thinking process adopted by humans when solving complex reasoning problems, that is, by iteratively decomposing the original problem into simpler sub - problems and gradually aggregating the answers of these sub - problems to solve the original problem. ### Main features of SOCRATIC QUESTIONING 1. **Recursive thinking**: Unlike CoT, SOCRATIC QUESTIONING explicitly navigates the thinking space, stimulates effective recursive thinking, and thus improves the robustness to errors in the reasoning process. 2. **Divide - and - conquer strategy**: This algorithm adopts a divide - and - conquer approach, decomposes complex problems into multiple sub - problems, and then solves these sub - problems through a bottom - up backtracking process, and finally solves the original problem. 3. **Self - Questioning module**: This is a core component that utilizes large - language models to actively pose and answer sub - problems that are crucial for solving the target problem. ### Experimental results The experimental results show that SOCRATIC QUESTIONING significantly outperforms existing prompting methods, such as CoT, Self - Consistency Chain - of - Thought (SC - CoT) and Tree - of - Thought (ToT), on multiple complex reasoning tasks. In particular, in multi - modal reasoning tasks, SOCRATIC QUESTIONING also performs well, being able to better handle visual information and generate accurate answers. ### Summary In general, this paper proposes a novel recursive - thinking algorithm - SOCRATIC QUESTIONING, which improves the accuracy and robustness of large - language models in solving complex reasoning problems by imitating the recursive thinking process of humans.

The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models

When Do Program-of-Thought Works for Reasoning?

Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Models

Supervised Chain of Thought

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Tree-of-Reasoning Question Decomposition for Complex Question Answering with Large Language Models

Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning

Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models

Break the Chain: Large Language Models Can be Shortcut Reasoners

Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models

Automatic Chain of Thought Prompting in Large Language Models

Large Language Models as Analogical Reasoners

Active Prompting with Chain-of-Thought for Large Language Models

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models

Self-Harmonized Chain of Thought

Ask more, know better: Reinforce-Learned Prompt Questions for Decision Making with Large Language Models

Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thought

Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic