Abstract:Reasoning encompasses two typical types: deductive reasoning and inductive reasoning. Despite extensive research into the reasoning capabilities of Large Language Models (LLMs), most studies have failed to rigorously differentiate between inductive and deductive reasoning, leading to a blending of the two. This raises an essential question: In LLM reasoning, which poses a greater challenge - deductive or inductive reasoning? While the deductive reasoning capabilities of LLMs, (i.e. their capacity to follow instructions in reasoning tasks), have received considerable attention, their abilities in true inductive reasoning remain largely unexplored. To investigate into the true inductive reasoning capabilities of LLMs, we propose a novel framework, SolverLearner. This framework enables LLMs to learn the underlying function (i.e., $y = f_w(x)$), that maps input data points $(x)$ to their corresponding output values $(y)$, using only in-context examples. By focusing on inductive reasoning and separating it from LLM-based deductive reasoning, we can isolate and investigate inductive reasoning of LLMs in its pure form via SolverLearner. Our observations reveal that LLMs demonstrate remarkable inductive reasoning capabilities through SolverLearner, achieving near-perfect performance with ACC of 1 in most cases. Surprisingly, despite their strong inductive reasoning abilities, LLMs tend to relatively lack deductive reasoning capabilities, particularly in tasks involving ``counterfactual'' reasoning.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to explore the capabilities of large language models (LLMs) in deductive reasoning and inductive reasoning. Specifically, the paper raises the following questions: 1. **Which is more challenging for LLMs: deductive reasoning or inductive reasoning?** - Although there has been extensive research on the reasoning capabilities of LLMs, most studies have failed to clearly distinguish between deductive reasoning and inductive reasoning, leading to confusion between the two. This raises a critical question: which is more challenging in LLM reasoning, deductive reasoning or inductive reasoning? 2. **Has the inductive reasoning capability of LLMs been fully explored?** - While the performance of LLMs on deductive reasoning tasks has received widespread attention, their true inductive reasoning capabilities have yet to be fully explored. To answer these questions, the paper proposes a new framework—**SolverLearner**, which enables LLMs to learn the mapping function from input data points to output values (i.e., $ y = f_w(x) $) solely through contextual examples. By focusing on inductive reasoning and separating it from deductive reasoning, it is possible to isolate and study the inductive reasoning capabilities of LLMs in a purely formal manner. ### Main Findings 1. **LLMs exhibit weaker capabilities in deductive reasoning, especially in "counterfactual" tasks.** - The study finds that LLMs show strong capabilities in standard tasks (such as decimal arithmetic) but perform poorly in "counterfactual" tasks. This is consistent with previous research findings. 2. **LLMs demonstrate exceptional inductive reasoning capabilities through SolverLearner.** - Using the SolverLearner framework, LLMs exhibit near-perfect performance in all test cases, especially when using GPT-4. This indicates that LLMs possess strong inductive reasoning capabilities. 3. **Deductive reasoning is more challenging for LLMs than inductive reasoning.** - By comparing the performance of pure deductive reasoning and pure inductive reasoning, the paper finds that LLMs face greater challenges in deductive reasoning tasks. In summary, this paper successfully separates the deductive and inductive reasoning capabilities of LLMs by introducing the SolverLearner framework and reveals the different performances of LLMs in these two aspects.

Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

LLMs for Relational Reasoning: How Far are We?

The Role of Deductive and Inductive Reasoning in Large Language Models

Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach

Evaluating the Deductive Competence of Large Language Models

Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning

Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement

Can LLMs Compute with Reasons?

When Do Program-of-Thought Works for Reasoning?

Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond

Eliminating Reasoning via Inferring with Planning: A New Framework to Guide LLMs' Non-linear Thinking

A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences

Are LLMs the Master of All Trades? : Exploring Domain-Agnostic Reasoning Skills of LLMs

Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs

Can LLMs Reason in the Wild with Programs?

Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs

Large Language Models Are In-Context Semantic Reasoners Rather Than Symbolic Reasoners

Can Large Language Models Reason? A Characterization via 3-SAT

Hypothesis Search: Inductive Reasoning with Language Models

A Closer Look at Logical Reasoning with LLMs: The Choice of Tool Matters